Gradle Test Executor process hangs and cannot be killed (Gradle 7.6.4)

Help/Discuss
1

Short Summary

When running tests with Gradle 7.6.4, sometimes a Gradle Test Executor process hangs and cannot be killed. jps does not list it, jstack cannot attach, and the process stays alive with many RocksDB and gRPC Netty threads. This prevents Gradle from creating the next Test Executor, causing the entire test task to freeze. Manually killing the stuck process allows Gradle to continue.

Detailed Description

Problem

  • During ./gradlew clean build, one of the Gradle Test Executor (n) processes gets stuck and cannot be terminated.
  • Once this happens, Gradle is unable to create the next Gradle Test Executor (n+1), causing the entire test task to hang.
  • Manual intervention is required: if we kill the stuck process manually, Gradle immediately spawns the next executor and continues running tests.

Symptoms

  • jps does not list the stuck Test Executor process.
  • ps shows the process is still alive.
  • jstack -l <pid> cannot attach to it.
  • The process contains many threads, including rocksdb:low, rocksdb:high, and gRPC/Netty event loop threads.

Example output from ps -L -p <pid> shows hundreds of active threads, including those for RocksDB and gRPC.

Test configuration

test {
    retry {
        maxRetries = 5
        maxFailures = 20
    }
    testLogging {
        exceptionFormat = 'full'
    }
    jacoco {
        destinationFile = file("$buildDir/jacoco/jacocoTest.exec")
        classDumpDir = file("$buildDir/jacoco/classpathdumps")
    }

    maxHeapSize = "1024m"
    doFirst {
        forkEvery = 100
        jvmArgs "-XX:MetaspaceSize=128m","-XX:MaxMetaspaceSize=256m", "-XX:+UseG1GC"
    }
}

gradle.properties:

org.gradle.parallel=true
org.gradle.jvmargs=-Xms1g

Logs

We see warnings from gRPC:

Previous channel ManagedChannelImpl{...} was garbage collected without being shut down!
Make sure to call shutdown()/shutdownNow()

Observations

  • The stuck executors appear related to native thread pools (RocksDB and gRPC Netty).
  • If resources are not properly shut down, the executor process never exits.
  • Because of this, Gradle is unable to proceed with the next Test Executor.

Questions

  1. Is this a known issue with Gradle 7.6.x Test Executors?
  2. Is there a recommended way to force Gradle to terminate orphaned Test Executor processes (e.g., org.gradle.test.worker.forceCloseProcesses)?
  3. Would upgrading to Gradle 8.x improve process cleanup in this scenario?
  4. Are there best practices for running tests with native resources (RocksDB, gRPC) to avoid blocking the Test Executor shutdown?

Workarounds

  • Manually killing the stuck executor allows the tests to continue.

:backhand_index_pointing_right: Any insights or guidance from the Gradle team/community would be greatly appreciated!