Html test report generated by gradle is really poor. For example - there are no skipped tests shown and passed tests are also not listed (only classes but not the tests themselves). I’m using gradle 1.6 with testng 6.8.1. Is there some plan to improve reports?
I don’t know if the report for TestNG is different or not but for JUnit, tests (including ignored) are listed. You just need to click on the test class name.
It doesn’t work with TestNG, please check build.gradle: https://gist.github.com/mawek/6000643 and test class: https://gist.github.com/mawek/6000649
Full source code: https://www.dropbox.com/s/km64wr5gmjv2xfx/gradle_logging_testng.zip
Generated html report contains no information about skipped tests.
The example project you have linked, seems to generate a good html report for me (even showing ignored). Just like for JUnit.
Also, I have checked that throwing ‘SkipException’ will be reported as ignored in the html report as well. However ‘@Test(enabled=false)’ is not shown in the html report.
Maybe we see something different:) This is what I get when I run ‘gradle test’:
Am I doing something wrong?
The classname in “Classes” (i.e., ‘com.mawek.test.Test01’) is a link to more detailed information. There you should see test methods of that particular class.
Oh I can see that now. It’s really burried down 
There is no way I can see that on main(index) page? I’d like to see on main page something like ‘2 tests completed, 1 failed, 1 skipped’.
The thing is that I’m trying to show this report on jenkins job main page, and it’s not clear that some test were skipped at all. I have to dig really deep to discover skipped tests.
Probably it would be nice if the number of skipped tests would be listed just like failed and succeeded.
I think, what you can do know is to manually parse the generated xml and count the ignored tests in a Gradle task.
EDIT: Although the number of skipped tests can be calculated from the “Tests” and “Failures” column.
Thanks for your time Attila. Can I somehow create feature request in gradle jira for this? I’d really like to see this on main page.
Calculating skipped tests does work but it’s not visible at the first sight, and parsing generated xml files or hooking into TestCountLogger seems like temporary hack to me.
As far as I know, you can no longer open jira tickets yourself. Instead, you should post RFE as an idea on the forum.
Ok thanks. By the way, I just found out that reportng reports do contain skipped tests in main page (but you can’t use it with ‘maxParallelForks’ switch in test task because it causes reports to overwrite themselves)
dependencies {
testCompile “org.testng:testng:6.8.1”
testCompile(“org.uncommons:reportng:1.1.2”) {
exclude module: ‘testng’
}
}
test {
useTestNG() {
useDefaultListeners = true // generate html report by TestNG
}
testReport = false // don’t generate test report by gradle
}
So you would like number of skipped tests on the front page?
TestNG’s HTML report stuff is incompatible with maxParallelForks. This is something we can’t fix.
Yes, that would be cool.
Also percentage successful is kind of misleading. If I have 10 tests, one of them passes, one fails and 8 (dependent) are skipped, I get 90% success rate.
I understand that skipped tests are not necessary failures, but when I see 90% success rate, I expect that most of the tests passed OK which is not really true
I, too, am in this same situation and because TestNG does not play well with parallel testing, I’m forced to use Gradle’s built-in reporting. Not having the skipped tests is tough to digest. I’d love to see this feature implemented.
Any plans to do so?
Raised as GRADLE-2976.
Awesome, I’m looking forward to this when you get a chance. Both me and my team will be quite pleased if and when this makes it in.
I just submitted a pull request to implement this improvement: https://github.com/gradle/gradle/pull/230
GRADLE-2976 has been resolved with jnizet’s pull request. It should be available from 1.11
And If I can add done more thing for the report. If you follow this example http://mrhaki.blogspot.com/2010/11/gradle-goodness-running-tests-in.html For parallel thread using and start it with 13 threads fore example one of the execution for my machine was:
BUILD SUCCESSFUL
Total time: 13.463 secs
But in the report was said the total time only
341 tests 0 failures 1m8.91s duration
Duration of sum time is over 1 minute, but not the whole execution. So must be also added information about the real time spend for tests run. Or I am wrong?