Success rate in summary to reflect features pass/fails

[jacattau]

Hi there,

I'm liking this plugin, however one gripe I have is in the top summary section is the way the stats are presented. For example, if I run one Spec file that has 10 features in it, and 9 pass and 1 fails, that is a 90% pass rate. This is reflected correctly in the Specifications portion on bottom. However in the top Summary section, total is 1, 0 passed, 1 failed, 0% success rate and the pie chart shows 0%.

Is there a reason you chose this all or nothing approach for the summary? I think it would be much more beneficial if it was summarized by features, so that it would show total is 10, 9 passed, 1 failed, 90% success rate. I ran one file that was 90% successful but the summary shows 0% successful and that seems misleading.

[trinidad8516]

I am glad you like it, just for clarification the Summary section has statistics at the Specification Level (Class) where the Specifications section has statistics at Feature Level (Method). May be in the near future this could be an enhancement :) .

[jacattau]

Thanks for the response. I understand what is happening, I guess I just see value in having the "total" and the pie chart for the specification level as having more value because it's more accurate/descriptive. I would be excited to see that as an enhancement!

[tilmanginzel]

Hi @jacattau, thanks for the feedback! I can see that this can be a bit annoying. Currently I am relying on the stats from spock-reports, but it should not be too difficult to also show the success rate on a feature level here.

I will keep this in mind for a future release! It might also makes sense to provide this feature directly in spock-reports.

[jacattau]

I just wanted to drop by to say I am still interested in this enhancement :)

[tilmanginzel]

Hey @jacattau, do you have an idea on how this could be shown in the UI, without adding too much clutter?

Proposal 1: Adding another column Feature success rate to the summary table. This shouldn't be too difficult, but then it is unclear what the pie chart on the right refers to. Adding a second chart wouldn't really fit well here.

Proposal 2: Have two separate summary tables, and two pie charts. One for the specs, and another one for the features. The chart is not shown in the image below, but just imagine a second one below. Dummy example:

What do you think?

[jacattau]

I believe option 2 is closest to what I am picturing. I believe the pie chart is showing the rate of the summary where I would rather have it show the rate of the specifications.

So using an example here. If I have 5 features each with 10 tests inside of them and I run all the tests and 1 test fails that means 49/50 (98%) passed. At the feature level that would say 4/5 passed (80%). What I would like to see is the 98% represented.

[tilmanginzel]

Yeah I agree that the 98% success rate is more representative and relevant than the 80% success rate. Just to clarify: I think you have mixed up specifications and features in your description. A single specification is a class which contains many features.

I am starting to wonder if those pie charts are really necessary, but I don't want to get rid of the visual representation entirely. Maybe a very simple stacked bar chart which can show both specification and feature success rate could be a compromise.

[jacattau]

You are right, I definitely had those mixed up. Must not have had my coffee yet :D

[ShubhenduPandey]

@tilmanginzel
Have you finished developing this? Our client requirement is to show fixture success rate on the report dashboard. Thank you in advance.

[tilmanginzel]

Hi @ShubhenduPandey, I would still prefer if those statistics are added in spock-reports directly, see the issue which is linked above your comment. I don't want to deviate too much from spock-reports, as this might increase the chance of breaking changes in the future which are harder to maintain.

In the meantime, if it is urgent for you, I would also accept pull requests to add this to geb-spock-reports until it is available in spock-reports.

[ShubhenduPandey]

@tilmanginzel Geb-spock-report doesn't work for me. Getting error below error

[(Could not resolve all dependencies for configuration ':testCompile'.

Could not find com.aoe:geb-spock-reports:0.2.6.
Searched in the following locations:
https://repo1.maven.org/maven2/com/aoe/geb-spock-reports/0.2.6/geb-spock-reports-0.2.6.pom
https://repo1.maven.org/maven2/com/aoe/geb-spock-reports/0.2.6/geb-spock-reports-0.2.6.jar
)]
I am currently using spock report.

[tilmanginzel]

@ShubhenduPandey geb-spock-reports is published on jcenter, you need to add it as a repository in your build.gradle:

repositories {
    jcenter()
}

[ShubhenduPandey]

Thanks @tilmanginzel Can you please merge pull requests to add this to geb-spock-reports.

[tilmanginzel]

Currently there are no pull request available to merge. Maybe there was a misunderstanding. I will accept a pull request if someone is willing to work on this, but I don't have the time to do it myself.

[ShubhenduPandey]

I misunderstood it. Just wanted to know how you have calculated the total feature count in the first place? I didn't see a method that calculated total features in Utils.stats and Utils.aggregateStats methods.

[tilmanginzel]

I didn't see a method that calculated total features in Utils.stats and Utils.aggregateStats methods.

Correct, that is why I proposed to add these statistics to spock-reports: renatoathaydes/spock-reports#193