Gradle caching fails, when tracking test coverage for functional plugin tests

Help/Discuss
1

Hello there,

we’ve been writing Gralde Plugins for our projects configurations for quiet a while now and did some functional testing allmost exactly how it is described here: https://docs.gradle.org/nightly/userguide/testing_gradle_plugins.html#setting_up_automated_tests

Using a dedicated source set and testing with spok works pretty nice for us.

We recently discovered, that we are missing the testcoverage from jacoco for these kind of tests, so we applied the https://github.com/koral--/jacoco-gradle-testkit-plugin plugin.
At first everything was ok, starting functional tests from Intellij workes like it should and we could also produce a html report afterwards to see that the coverage was recorded successfully.

But then we found out that on a windows (developer) system we can no longer execute those tests from the command line, and this is how it fails:

Failed to load cache entry for task ':resource-validator-plugin:functionalTest', cleaning outputs and falling back to (non-incremental) execution
org.gradle.api.UncheckedIOException: Failed to create MD5 hash for file content.
        at org.gradle.internal.hash.DefaultStreamHasher.hash(DefaultStreamHasher.java:37)
        at org.gradle.internal.hash.DefaultFileHasher.hash(DefaultFileHasher.java:40)
        at org.gradle.api.internal.changedetection.state.CachingFileHasher.snapshot(CachingFileHasher.java:93)
        at org.gradle.api.internal.changedetection.state.CachingFileHasher.hash(CachingFileHasher.java:71)
        at org.gradle.api.internal.changedetection.state.SplitFileHasher.hash(SplitFileHasher.java:65)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter.snapshot(DefaultFileSystemSnapshotter.java:163)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter.snapshotAndCache(DefaultFileSystemSnapshotter.java:149)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter.snapshotAndCache(DefaultFileSystemSnapshotter.java:133)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter.access$200(DefaultFileSystemSnapshotter.java:62)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter$2.create(DefaultFileSystemSnapshotter.java:116)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter$2.create(DefaultFileSystemSnapshotter.java:113)
        at org.gradle.cache.internal.ProducerGuard$StripedProducerGuard.guardByKey(ProducerGuard.java:115)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter.snapshot(DefaultFileSystemSnapshotter.java:113)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter$FileCollectionLeafVisitorImpl.visitCollection(DefaultFileSystemSnapshotter.java:241)
        at org.gradle.api.internal.file.AbstractFileCollection.visitLeafCollections(AbstractFileCollection.java:233)
        at org.gradle.api.internal.file.CompositeFileCollection.visitLeafCollections(CompositeFileCollection.java:205)
        at org.gradle.internal.snapshot.impl.DefaultFileSystemSnapshotter.snapshot(DefaultFileSystemSnapshotter.java:126)
        at org.gradle.internal.fingerprint.impl.AbstractFileCollectionFingerprinter.fingerprint(AbstractFileCollectionFingerprinter.java:48)
        at org.gradle.api.internal.tasks.execution.DefaultTaskFingerprinter.fingerprintTaskFiles(DefaultTaskFingerprinter.java:45)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.snapshotAfterOutputsGenerated(ExecuteActionsTaskExecuter.java:328)
        at org.gradle.internal.execution.steps.SnapshotOutputsStep.execute(SnapshotOutputsStep.java:48)
        at org.gradle.internal.execution.steps.SnapshotOutputsStep.execute(SnapshotOutputsStep.java:31)
        at org.gradle.internal.execution.steps.CacheStep.executeWithoutCache(CacheStep.java:201)
        at org.gradle.internal.execution.steps.CacheStep.executeAndStoreInCache(CacheStep.java:183)
        at org.gradle.internal.execution.steps.CacheStep.lambda$executeWithCache$2(CacheStep.java:106)
        at java.util.Optional.orElseGet(Optional.java:267)
        at org.gradle.internal.execution.steps.CacheStep.lambda$executeWithCache$3(CacheStep.java:106)
        at org.gradle.internal.Try$1.apply(Try.java:59)
        at org.gradle.internal.Try$1.apply(Try.java:56)
        at org.gradle.internal.Try$Success.flatMap(Try.java:102)
        at org.gradle.internal.Try.map(Try.java:56)
        at org.gradle.internal.execution.steps.CacheStep.executeWithCache(CacheStep.java:79)
        at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:69)
        at org.gradle.internal.execution.steps.CacheStep.execute(CacheStep.java:45)
        at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:49)
        at org.gradle.internal.execution.steps.StoreSnapshotsStep.execute(StoreSnapshotsStep.java:43)
        at org.gradle.internal.execution.steps.StoreSnapshotsStep.execute(StoreSnapshotsStep.java:32)
        at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:38)
        at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:24)
        at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:96)
        at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$0(SkipUpToDateStep.java:89)
        at java.util.Optional.map(Optional.java:215)
        at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:54)
        at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:38)
        at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:77)
        at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:37)
        at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:36)
        at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:26)
        at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:90)
        at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:48)
        at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:33)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:117)
        at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionStateTaskExecuter.execute(ResolveBeforeExecutionStateTaskExecuter.java:75)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:108)
        at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
        at org.gradle.api.internal.tasks.execution.StartSnapshotTaskInputsBuildOperationTaskExecuter.execute(StartSnapshotTaskInputsBuildOperationTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:94)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:95)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:73)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:49)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:49)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:134)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:129)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: The process cannot access the file because another process has locked a portion of the file
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:233)
        at org.gradle.internal.hash.DefaultStreamHasher.doHash(DefaultStreamHasher.java:52)
        at org.gradle.internal.hash.DefaultStreamHasher.hash(DefaultStreamHasher.java:35)
        ... 90 more

We already tried to force sequential execution of tests with

functionalTest {
    maxParallelForks = 1
}

but had no success.

We also tried gradlew functionalTest --no-daemon --no-build-cache without success.

On our Linux based build environment, this is not a problem, only on the developers windows machines.
We are using Gradle 5.5.1 (a update is scheduled, but will take some time) and Java 1.8.0_202.

Any ideas are appreciated.

Update:
Tested with amazon-corretto-8.275.01.1-windows-x64-jdk (1.8.0_275) and still failing.

another possibly relevant output:

Custom actions are attached to task ':resource-validator-plugin:functionalTest'.
Caching disabled for task ':resource-validator-plugin:functionalTest' because:
  Gradle does not know how file 'build\jacoco\functionalTest.exec' was created (output property 'jvmArgumentProviders.jacocoAgent$0.jacoco.destinationFile'). Task output caching requires exclusive access to output paths to guarantee correctness.

maybe the binary jacoco.exec file is beeing written to twice - from the jacocoAgent of the calling gradle instance and from the jacocoAgent of the forked test build? Not sure how to suppress that and why it is only failing on windows?

2

Seems I found a solution:

        // the usual configuration
        project.apply plugin: 'pl.droidsonroids.jacoco.testkit'
        jacocoTestKit {
            applyTo("functionalTestRuntimeOnly", functionalTestTask)
        }
        // write the jacoco.exec to another file than the gradle functionTest jacoco agent does
        tasks.named("generateJacocoFunctionalTestKitProperties") {
            destinationFile = "${buildDir}/jacoco/testkit.exec"
        }
        // reporting task is required for sonarqube (will drop support for exec files)
        // attention: this will force you to have tests, that write the testkit.exec file or the reporting will fail!
        def jacocoFunctionalTestReportTask = tasks.register("jacocoFunctionalTestReport", JacocoReport) {
            description = 'Generate jacoco report for functionalTest coverage'

            mustRunAfter functionalTestTask
            executionData ("${buildDir}/jacoco/functionalTest.exec", "${buildDir}/jacoco/testkit.exec")
            sourceSets sourceSets.main
        }
3

After upgrading to gradle 6.8.1 the issue reappeared and not only on windows.
This time I found a issue of another user who’s just waiting until the file is not locked anymore. If someone else comes across the same problem that the testkit engine is not fully stopped after the tests finished, this might help you too, even if it’s not for the testcoverage you’re looking for.