Daemon stuck: "Waiting to acquire shared lock on daemon addresses registry"

not-a-bug

(Tony Robalik) #1

I just updated my Android project to use Gradle 2.14.1 (from 2.14), and also the Android Gradle build tools to 2.1.3 (from 2.1.2). I then tried to build with the following command:

./gradlew clean assembleDebugAndroidTest

It gets stuck on this line:

> Building 91% > :app:transformClassesWithDexForDebug

When I run with the --info and --debug flags, I see the following output:

12:34:28.756 [INFO] [org.gradle.api.Project] ignored resource META-INF/LICENSE
12:34:28.756 [INFO] [org.gradle.api.Project] ignored resource META-INF/MANIFEST.MF
12:34:28.756 [INFO] [org.gradle.api.Project] ignored resource META-INF/NOTICE
12:34:28.756 [INFO] [org.gradle.api.Project] ignored resource META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.properties
12:34:28.756 [INFO] [org.gradle.api.Project] ignored resource META-INF/maven/com.fasterxml.jackson.core/jackson-databind/pom.xml
12:34:28.756 [INFO] [org.gradle.api.Project] ignored resource META-INF/services/com.fasterxml.jackson.core.ObjectCodec
12:34:28.756 [INFO] [org.gradle.api.Project] Dexing /Users/tonyrobalik/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.4.3/feff63199be7b8f495c2f3e2096dcb6bd5e5b0b3/jackson-databind-2.4.3.jar took 2.144 s.
12:34:50.586 [DEBUG] [org.gradle.launcher.daemon.server.Daemon] DaemonExpirationPeriodicCheck running
12:34:50.587 [DEBUG] [org.gradle.launcher.daemon.server.health.DaemonStatus] GC rate: 0.0/s
12:34:50.588 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
12:34:50.588 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired.
12:34:50.589 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
12:35:50.583 [DEBUG] [org.gradle.launcher.daemon.server.Daemon] DaemonExpirationPeriodicCheck running
12:35:50.583 [DEBUG] [org.gradle.launcher.daemon.server.health.DaemonStatus] GC rate: 0.0/s
12:35:50.584 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
12:35:50.584 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired.
12:35:50.584 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
> Building 93% > :app:transformClassesWithDexForUsDebug

If I sit and wait, I see it re-acquire and re-release the lock over and over again. Nothing I’ve done can resolve the problem. I even tried deleting my $HOME/.gradle/ directory for a fresh start – nothing.

Does anyone have any idea what’s happening?


(Gary Hale) #2

The message you are seeing is related to the daemon’s health check running in the background. You’ll notice that it runs on a consistent interval and finishes rather quickly. This is normal and not actually related to your problem. This issue appears to be something to do with the dexing task itself. You can see from the output that the garbage collector isn’t doing much (i.e. “GC rate: 0.0/s”) so it doesn’t appear like a memory-related issue. Probably the best way to diagnose it would be to get stacktrace (using jstack or something similar) and see what the dexing task is doing while its in this state.


(Tony Robalik) #3

Thanks for the reply. I ran the jps command:

jps
8163 GradleWrapperMain
468 
8230 Jps
7996 GradleDaemon

I assume it’s the PID of GradleWrapperMain I want, so next I ran jstack:

jstack 8163
2016-08-21 15:24:42
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.92-b14 mixed mode):

"Attach Listener" #15 daemon prio=9 os_prio=31 tid=0x00007fca46040800 nid=0x1307 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"pool-1-thread-1" #14 prio=5 os_prio=31 tid=0x00007fca44385000 nid=0x5a03 waiting on condition [0x00007000017db000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000076eb6a930> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"forward input" #13 prio=5 os_prio=31 tid=0x00007fca45315800 nid=0x5803 waiting on condition [0x00007000016d8000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000076eb00730> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at org.gradle.util.DisconnectableInputStream.read(DisconnectableInputStream.java:138)
	at org.gradle.launcher.daemon.client.InputForwarder$1.run(InputForwarder.java:77)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"DisconnectableInputStream source reader" #12 daemon prio=5 os_prio=31 tid=0x00007fca44384000 nid=0x5603 runnable [0x00007000015d5000]
   java.lang.Thread.State: RUNNABLE
	at java.io.FileInputStream.readBytes(Native Method)
	at java.io.FileInputStream.read(FileInputStream.java:255)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
	- locked <0x000000076eb1a1c8> (a java.io.BufferedInputStream)
	at org.gradle.util.DisconnectableInputStream$1.run(DisconnectableInputStream.java:96)
	at java.lang.Thread.run(Thread.java:745)

"Service Thread" #9 daemon prio=9 os_prio=31 tid=0x00007fca4403e800 nid=0x5203 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread3" #8 daemon prio=9 os_prio=31 tid=0x00007fca45016000 nid=0x5003 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread2" #7 daemon prio=9 os_prio=31 tid=0x00007fca46007000 nid=0x4e03 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #6 daemon prio=9 os_prio=31 tid=0x00007fca43844000 nid=0x4c03 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=31 tid=0x00007fca44033800 nid=0x4a03 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=31 tid=0x00007fca44030000 nid=0x3e0f runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007fca4383a800 nid=0x3803 in Object.wait() [0x0000700000d3a000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000076eb08fb8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
	- locked <0x000000076eb08fb8> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007fca43837800 nid=0x3603 in Object.wait() [0x0000700000c37000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000076eb20178> (a java.lang.ref.Reference$Lock)
	at java.lang.Object.wait(Object.java:502)
	at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
	- locked <0x000000076eb20178> (a java.lang.ref.Reference$Lock)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

"main" #1 prio=5 os_prio=31 tid=0x00007fca45001800 nid=0x1703 runnable [0x0000700000217000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x000000076eb28250> (a sun.nio.ch.Util$2)
	- locked <0x000000076eb28260> (a java.util.Collections$UnmodifiableSet)
	- locked <0x000000076eb28160> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
	at org.gradle.internal.remote.internal.inet.SocketConnection$SocketInputStream.read(SocketConnection.java:165)
	at com.esotericsoftware.kryo.io.Input.fill(Input.java:139)
	at com.esotericsoftware.kryo.io.Input.require(Input.java:159)
	at com.esotericsoftware.kryo.io.Input.readInt(Input.java:308)
	at org.gradle.internal.serialize.kryo.KryoBackedDecoder.readSmallInt(KryoBackedDecoder.java:120)
	at org.gradle.internal.serialize.DefaultSerializerRegistry$TaggedTypeSerializer.read(DefaultSerializerRegistry.java:105)
	at org.gradle.internal.serialize.Serializers$StatefulSerializerAdapter$1.read(Serializers.java:36)
	at org.gradle.internal.remote.internal.inet.SocketConnection.receive(SocketConnection.java:78)
	at org.gradle.launcher.daemon.client.DaemonClientConnection.receive(DaemonClientConnection.java:75)
	at org.gradle.launcher.daemon.client.DaemonClientConnection.receive(DaemonClientConnection.java:35)
	at org.gradle.launcher.daemon.client.DaemonClient.monitorBuild(DaemonClient.java:194)
	at org.gradle.launcher.daemon.client.DaemonClient.executeBuild(DaemonClient.java:162)
	at org.gradle.launcher.daemon.client.DaemonClient.execute(DaemonClient.java:125)
	at org.gradle.launcher.daemon.client.DaemonClient.execute(DaemonClient.java:80)
	at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:43)
	at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:173)
	at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:239)
	at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:212)
	at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
	at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
	at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
	at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
	at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:205)
	at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
	at org.gradle.launcher.Main.doAction(Main.java:33)
	at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:55)
	at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:36)
	at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
	at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:129)
	at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)

"VM Thread" os_prio=31 tid=0x00007fca46030000 nid=0x3403 runnable 

"GC task thread#0 (ParallelGC)" os_prio=31 tid=0x00007fca4500d800 nid=0x2403 runnable 

"GC task thread#1 (ParallelGC)" os_prio=31 tid=0x00007fca4500e000 nid=0x2603 runnable 

"GC task thread#2 (ParallelGC)" os_prio=31 tid=0x00007fca4500e800 nid=0x2803 runnable 

"GC task thread#3 (ParallelGC)" os_prio=31 tid=0x00007fca43810800 nid=0x2a03 runnable 

"GC task thread#4 (ParallelGC)" os_prio=31 tid=0x00007fca43811000 nid=0x2c03 runnable 

"GC task thread#5 (ParallelGC)" os_prio=31 tid=0x00007fca43811800 nid=0x2e03 runnable 

"GC task thread#6 (ParallelGC)" os_prio=31 tid=0x00007fca43812800 nid=0x3003 runnable 

"GC task thread#7 (ParallelGC)" os_prio=31 tid=0x00007fca43813000 nid=0x3203 runnable 

"VM Periodic Task Thread" os_prio=31 tid=0x00007fca44031000 nid=0x5403 waiting on condition 

JNI global references: 298

Honestly, I’m not really sure how to parse this. Any ideas?

The gradle command I ran to achieve this state was: ./gradlew installDebugAndStart, which is a custom task that is defined thus:

task installDebugAndStart(type: Exec) {
    dependsOn 'installDebug'
    commandLine "${project.android.adbExe} shell am start com.project/com.project.Activity".split(' ')
}

Basically, it installs the APK and then launches it.


(Gary Hale) #4

Actually, you want a stacktrace from the “GradleDaemon” process.


(Sameer Jethvani) #5

I am also facing simillar issue , where gradle s not releasing lock on shared file

Have followed so many posts like below , but no luck

https://issues.gradle.org/browse/GRADLE-2795


(Tony Robalik) #6

Ah, thank you and my apologies. I appreciate your help – I have no idea what I should be looking for.

The stack trace is actually too long to paste here, so I’ve thrown it up on hastebin.


(Gary Hale) #7

So the very first thread (line 18) is likely the culprit. Its down in com.android.dx.rop.code.LocalVariableExtractor in the dexing code. Taking a quick glance at that class I see there’s a for loop in there - possibly it’s not exiting properly? In any event, the issue is outside of Gradle (and my expertise) - I’d raise this with the Android tools team (and share this stack trace with them).


(Tony Robalik) #8

Will do. Thank you very much.


(Tony Robalik) #9

For completeness’ sake, here is the link to the bug report I’ve filed with the Android tools team: https://code.google.com/p/android/issues/detail?id=220611&thanks=220611&ts=1471888230


(Sameer Jethvani) #10

Hey Gary_Hale

Glad to know that you guys were able to troubleshot trobalik’s issue :slight_smile:

However , I am still facing this hang issue , I have captured stacktrace of GradleWrapperMain process . It would be nice if you can look this jstack & suggest me any solution for this


(Sameer Jethvani) #11

Update
bebugging above JStack , I came to know that it was [org.eclipse.jgit] causing problem …

I was using old dependency of [org.ajoberstar], which in turn was using old version of above [org.eclipse.jgit] dependency .
Upgrading version of [org.ajoberstar] solved my problem