Gradle-1.1: "Timeout after waiting 120.0 seconds for Gradle Worker N to connect."

Old Forum Archive
1

Hi There,

After upgrading to Gradle-1.1, we get the following error sporadicly in our Jenkins nightly builds’ unit tests. We have about half a dozen Gradle projects running on that jenkins instance and the error can occur in each of them, but whether it occurs or not doesn’t follow a recognizable pattern…

  :my-project:test  Cobertura 1.9.4.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file  Instrumenting 1 file  Cobertura: Saved information on 58 classes.  Instrument time: 171ms  Flushing results...  Flushing results done  Cobertura: Loaded information on 58 classes.  Cobertura: Saved information on 58 classes.

 FAILURE: Build failed with an exception.

 * What went wrong:  Execution failed for task ':my-project:test'.  > Timeout after waiting 120.0 seconds for Gradle Worker 6 to connect.

 * Try:  Run with --info or --debug option to get more log output.

 * Exception is:  org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':my-project:test'.

at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:68)

at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)

at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:34)

at org.gradle.api.internal.changedetection.CacheLockHandlingTaskExecuter$1.run(CacheLockHandlingTaskExecuter.java:34)

at org.gradle.cache.internal.DefaultCacheAccess$2.create(DefaultCacheAccess.java:200)

at org.gradle.cache.internal.DefaultCacheAccess.longRunningOperation(DefaultCacheAccess.java:172)

at org.gradle.cache.internal.DefaultCacheAccess.longRunningOperation(DefaultCacheAccess.java:198)

at org.gradle.cache.internal.DefaultPersistentDirectoryStore.longRunningOperation(DefaultPersistentDirectoryStore.java:137)

at org.gradle.api.internal.changedetection.DefaultTaskArtifactStateCacheAccess.longRunningOperation(DefaultTaskArtifactStateCacheAccess.java:83)

at org.gradle.api.internal.changedetection.CacheLockHandlingTaskExecuter.execute(CacheLockHandlingTaskExecuter.java:32)

at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:55)

at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:57)

at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:41)

at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:51)

at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:52)

at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:42)

at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:247)

at org.gradle.execution.DefaultTaskGraphExecuter.executeTask(DefaultTaskGraphExecuter.java:192)

at org.gradle.execution.DefaultTaskGraphExecuter.doExecute(DefaultTaskGraphExecuter.java:177)

at org.gradle.execution.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:83)

at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:36)

at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)

at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)

at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:67)

at org.gradle.api.internal.changedetection.TaskCacheLockHandlingBuildExecuter$1.run(TaskCacheLockHandlingBuildExecuter.java:31)

at org.gradle.cache.internal.DefaultCacheAccess$1.create(DefaultCacheAccess.java:111)

at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:126)

at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:109)

at org.gradle.cache.internal.DefaultPersistentDirectoryStore.useCache(DefaultPersistentDirectoryStore.java:129)

at org.gradle.api.internal.changedetection.DefaultTaskArtifactStateCacheAccess.useCache(DefaultTaskArtifactStateCacheAccess.java:79)

at org.gradle.api.internal.changedetection.TaskCacheLockHandlingBuildExecuter.execute(TaskCacheLockHandlingBuildExecuter.java:29)

at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)

at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)

at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:67)

at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)

at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)

at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:54)

at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:155)

at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:110)

at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:78)

at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:38)

at org.gradle.launcher.exec.InProcessGradleLauncherActionExecuter.execute(InProcessGradleLauncherActionExecuter.java:39)

at org.gradle.launcher.exec.InProcessGradleLauncherActionExecuter.execute(InProcessGradleLauncherActionExecuter.java:25)

at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:50)

at org.gradle.launcher.cli.ActionAdapter.execute(ActionAdapter.java:30)

at org.gradle.launcher.cli.ActionAdapter.execute(ActionAdapter.java:22)

at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:200)

at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:173)

at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)

at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:138)

at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)

at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)

at org.gradle.launcher.Main.doAction(Main.java:48)

at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)

at org.gradle.launcher.Main.main(Main.java:39)

at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:50)

at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:32)

at org.gradle.launcher.GradleMain.main(GradleMain.java:26)  Caused by: org.gradle.process.internal.ExecException: Timeout after waiting 120.0 seconds for Gradle Worker 6 to connect.

at org.gradle.process.internal.DefaultWorkerProcess.start(DefaultWorkerProcess.java:114)

at org.gradle.api.internal.tasks.testing.worker.ForkingTestClassProcessor.processTestClass(ForkingTestClassProcessor.java:63)

at org.gradle.api.internal.tasks.testing.processors.RestartEveryNTestClassProcessor.processTestClass(RestartEveryNTestClassProcessor.java:45)

at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)

at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)

at org.gradle.messaging.dispatch.FailureHandlingDispatch.dispatch(FailureHandlingDispatch.java:29)

at org.gradle.messaging.dispatch.AsyncDispatch.dispatchMessages(AsyncDispatch.java:132)

at org.gradle.messaging.dispatch.AsyncDispatch.access$000(AsyncDispatch.java:33)

at org.gradle.messaging.dispatch.AsyncDispatch$1.run(AsyncDispatch.java:72)

at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:66)



BUILD FAILED

As far as I remember, that didn’t happen before 1.1 (but it might be a coincidence, or there might be another reason why it startet happening now).

Any idea or advice?

Best regards, Mike

2

** Update: Gradle 1.2 doesn’t solve the problem.

3

Can you provide more information, like the output of ‘gradle -v’ (in the environment where the problem occurs)? Can you also provoke this error in a minimal build which, say, only runs a couple of tests, or does it only occur in your production builds? Also, please run with debug logging and post the result to gist.github.com.

4

Hi Peter,

It happens on our jenkins build server, so I created a test job with the same config as our production build and let it execute gradle -v, here’s the full output:

Building in workspace /local/jenkins/jenkins_workspace/jobs/test_job/workspace
[workspace] $ /local/jenkins/gradle-1.2/bin/gradle -v
Please use CMSClassUnloadingEnabled in place of CMSPermGenSweepingEnabled in the future
  ------------------------------------------------------------
Gradle 1.2
------------------------------------------------------------
  Gradle build time: Wednesday, September 12, 2012 10:46:02 AM UTC
Groovy: 1.8.6
Ant: Apache Ant(TM) version 1.8.4 compiled on May 22 2012
Ivy: 2.2.0
JVM: 1.6.0_29 (Sun Microsystems Inc. 20.4-b02)
OS: Linux 2.6.16.21-0.8-smp amd64
  Build step 'Invoke Gradle script' changed build result to SUCCESS
Finished: SUCCESS

It looks like it only happens in our production builds. They typically involve some code generation and unit testing (including unit testing against a real oracle database). We also use checkstyle and cobertura on them. Since it looks like it only happens on our bigger production projects, I’m afraid I can’t just post a full debug log, and it’s too much debug output to strip out company-internal stuff… Is there something in particular you’re looking for? If so, maybe I could extract just that portion for you?

5

I’m just trying to gather as much information as possible. Without a way to reproduce this, it will be very hard to fix. It’s likely somehow related to your environment. We never see this in any of our builds, and we run them in a number of different environments.

6

If you run with --debug we would be interested only with the log entries that happen immediately before the problem occurs. Also, is it possible if you configure one of the jobs to use 1.0 and tell us if it works ok?

7

I’d like to give a quick feedback about this issue. I’ve been on vacation for 2 weeks now, and I just reviewed our CI build history.

It seems the issue “magically” stopped occurring about 3 weeks ago, so I also suspect our corporate IT infrastructure to be the cause somehow. Of course, now that I know what to look for in the logs, I can’t reproduce it anymore :frowning:

Anyhow, if it ever starts happening again, I’ll post back here, but for now it seems it just works.

Thanks, as always, for your kind support :slight_smile: - Mike

8

It happened again today, but this time, I had debug enabled.

I tried to extract all info about the concerned “gradle worker 6” and put it together while doing my best not to leave some company internals in there :wink:

Here are the debug log snippets: https://gist.github.com/4000098

Let me know if you need further log file grepping :wink:

Best regards, Mike

9

Hi Mike,

I had exactly the same issue as you. Have you figured out your problem? If yes can you share some information here?

Thanks, Kevin

10

Unfortunately not, we’re still having the same issue with the latest gradle and a reinstalled CentOS 6. It happens so rarely though, that I just re-run failed builds manually.

11

The timeout issue is gone when I removed some unnecessary runtime classpaths. So it seems either there is a limitation on the max length of classpath or it’s really timeout on the huge classpath (not clear what’s under the hood)? If it’s really a timeout, can gradle provide api to customize it? Thanks.