Hi–
As a long time Gradle user, I thought that the right way on a build server was to not use the daemon – I didn’t realize that it was changed to always run one.
So how do I troubleshoot what is going on with it?
The logs look like:
2019-11-10T21:46:58.636-0800 [DEBUG] [org.gradle.launcher.daemon.bootstrap.DaemonMain] Assuming the daemon was started with following jvm opts: [–add-opens, java.base/java.util=ALL-UNNAMED, --add-opens, java.base/java.lang=ALL-UNNAMED, --add-opens, java.base/java.lang.invoke=ALL-UNNAMED, --add-opens, java.prefs/java.util.prefs=ALL-UNNAMED, -Xms4G, -Xmx4G, -Dfile.encoding=UTF-8, -Duser.country=US, -Duser.language=en, -Duser.variant]
2019-11-10T21:46:58.819-0800 [INFO] [org.gradle.launcher.daemon.server.Daemon] start() called on daemon - DefaultDaemonContext[uid=b19cf6d4-1a4e-46c1-a6fe-56312b78e7fe,javaHome=/usr/lib/jvm/java-11-openjdk-amd64,daemonRegistryDir=/Local/jenkins/.gradle/daemon,pid=24081,idleTimeout=10800000,priority=NORMAL,daemonOpts=–add-opens,java.base/java.util=ALL-UNNAMED,–add-opens,java.base/java.lang=ALL-UNNAMED,–add-opens,java.base/java.lang.invoke=ALL-UNNAMED,–add-opens,java.prefs/java.util.prefs=ALL-UNNAMED,-Xms4G,-Xmx4G,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]
2019-11-10T21:46:58.844-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface veth5437ffa
2019-11-10T21:46:58.845-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.845-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:1815:c3ff:fef9:8583%veth5437ffa
2019-11-10T21:46:58.846-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface vethc64b768
2019-11-10T21:46:58.846-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.849-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:d88d:4ff:fe29:aa20%vethc64b768
2019-11-10T21:46:58.849-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface veth05b51fa
2019-11-10T21:46:58.850-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.850-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:3cce:a3ff:fe8e:7d8e%veth05b51fa
2019-11-10T21:46:58.850-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface veth1089372
2019-11-10T21:46:58.851-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.851-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:50bb:34ff:fe1a:f0fe%veth1089372
2019-11-10T21:46:58.851-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface vethef4509a
2019-11-10T21:46:58.852-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.852-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:c00d:abff:fe1b:b8a6%vethef4509a
2019-11-10T21:46:58.852-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface tun0
2019-11-10T21:46:58.853-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.853-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:d09a:7193:76ba:2e61%tun0
2019-11-10T21:46:58.853-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /192.168.3.5
2019-11-10T21:46:58.853-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface docker0
2019-11-10T21:46:58.854-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.854-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:42:f5ff:fed7:f988%docker0
2019-11-10T21:46:58.854-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /172.17.0.1
2019-11-10T21:46:58.855-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface eno2
2019-11-10T21:46:58.855-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.855-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /4545:0:0:0:1a66:daff:fe91:6cab%eno2
2019-11-10T21:46:58.855-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:1a66:daff:fe91:6cab%eno2
2019-11-10T21:46:58.856-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /192.168.0.100
2019-11-10T21:46:58.856-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface eno1
2019-11-10T21:46:58.856-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? false
2019-11-10T21:46:58.856-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /fe80:0:0:0:1a66:daff:fe91:6caa%eno1
2019-11-10T21:46:58.857-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /192.168.1.99
2019-11-10T21:46:58.857-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /192.168.1.164
2019-11-10T21:46:58.857-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /192.168.1.6
2019-11-10T21:46:58.858-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /192.168.1.5
2019-11-10T21:46:58.858-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /192.168.3.5
2019-11-10T21:46:58.858-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding remote address /192.168.1.1
2019-11-10T21:46:58.858-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding IP addresses for network interface lo
2019-11-10T21:46:58.859-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Is this a loopback interface? true
2019-11-10T21:46:58.859-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding loopback address /0:0:0:0:0:0:0:1%lo
2019-11-10T21:46:58.859-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.InetAddresses] Adding loopback address /127.0.0.1
2019-11-10T21:46:58.862-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.TcpIncomingConnector] Listening on [6d595694-d7f4-47f7-a87f-073af162da26 port:45991, addresses:[/0:0:0:0:0:0:0:1%lo, /127.0.0.1]].
2019-11-10T21:46:58.872-0800 [DEBUG] [org.gradle.launcher.daemon.server.Daemon] Daemon starting at: Sun Nov 10 21:46:58 PST 2019, with address: [6d595694-d7f4-47f7-a87f-073af162da26 port:45991, addresses:[/0:0:0:0:0:0:0:1%lo, /127.0.0.1]]
2019-11-10T21:46:58.873-0800 [INFO] [org.gradle.launcher.daemon.server.DaemonRegistryUpdater] Advertising the daemon address to the clients: [6d595694-d7f4-47f7-a87f-073af162da26 port:45991, addresses:[/0:0:0:0:0:0:0:1%lo, /127.0.0.1]]
2019-11-10T21:46:58.873-0800 [DEBUG] [org.gradle.launcher.daemon.server.DaemonRegistryUpdater] Advertised daemon context: DefaultDaemonContext[uid=b19cf6d4-1a4e-46c1-a6fe-56312b78e7fe,javaHome=/usr/lib/jvm/java-11-openjdk-amd64,daemonRegistryDir=/Local/jenkins/.gradle/daemon,pid=24081,idleTimeout=10800000,priority=NORMAL,daemonOpts=–add-opens,java.base/java.util=ALL-UNNAMED,–add-opens,java.base/java.lang=ALL-UNNAMED,–add-opens,java.base/java.lang.invoke=ALL-UNNAMED,–add-opens,java.prefs/java.util.prefs=ALL-UNNAMED,-Xms4G,-Xmx4G,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]
2019-11-10T21:46:58.875-0800 [DEBUG] [org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Storing daemon address: [6d595694-d7f4-47f7-a87f-073af162da26 port:45991, addresses:[/0:0:0:0:0:0:0:1%lo, /127.0.0.1]], context: DefaultDaemonContext[uid=b19cf6d4-1a4e-46c1-a6fe-56312b78e7fe,javaHome=/usr/lib/jvm/java-11-openjdk-amd64,daemonRegistryDir=/Local/jenkins/.gradle/daemon,pid=24081,idleTimeout=10800000,priority=NORMAL,daemonOpts=–add-opens,java.base/java.util=ALL-UNNAMED,–add-opens,java.base/java.lang=ALL-UNNAMED,–add-opens,java.base/java.lang.invoke=ALL-UNNAMED,–add-opens,java.prefs/java.util.prefs=ALL-UNNAMED,-Xms4G,-Xmx4G,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]
2019-11-10T21:46:58.884-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on daemon addresses registry.
2019-11-10T21:46:58.887-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2019-11-10T21:46:58.892-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2019-11-10T21:46:58.893-0800 [LIFECYCLE] [org.gradle.launcher.daemon.server.Daemon] Daemon server started.
2019-11-10T21:46:58.894-0800 [DEBUG] [org.gradle.launcher.daemon.bootstrap.DaemonStartupCommunication] Completed writing the daemon greeting. Closing streams…
2019-11-10T21:46:58.899-0800 [DEBUG] [org.gradle.launcher.daemon.server.Daemon] stopOnExpiration() called on daemon
2019-11-10T21:46:58.900-0800 [DEBUG] [org.gradle.launcher.daemon.server.Daemon] awaitExpiration() called on daemon
2019-11-10T21:46:58.901-0800 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon is running. Sleeping until state changes.
2019-11-10T21:46:58.902-0800 [ERROR] [org.gradle.internal.remote.internal.inet.TcpIncomingConnector] Cannot accept connection from remote address /192.168.1.1.
2019-11-10T21:47:08.910-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2019-11-10T21:47:08.910-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2019-11-10T21:47:08.911-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
2019-11-10T21:47:08.912-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2019-11-10T21:47:08.912-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2019-11-10T21:47:08.913-0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.
The last couple of lines repeat until its done.
$ ./gradlew --daemon clean assemble
Starting a Gradle Daemon, 4 busy Daemons could not be reused, use --status for details
Starting a Gradle Daemon, 5 busy Daemons could not be reused, use --status for details
Starting a Gradle Daemon, 6 busy Daemons could not be reused, use --status for details
Starting a Gradle Daemon, 7 busy Daemons could not be reused, use --status for details
$ ./gradlew --daemon clean assemble --status
PID STATUS INFO
23551 UNKNOWN UNKNOWN
23580 UNKNOWN UNKNOWN
26789 UNKNOWN UNKNOWN
24052 UNKNOWN UNKNOWN
26821 UNKNOWN UNKNOWN
26853 UNKNOWN UNKNOWN
24081 UNKNOWN UNKNOWN
Only Daemons for the current Gradle version are displayed. See The Gradle Daemon
ps just shows those processes:
jenkins 23551 1 1 21:45 ? 00:00:06 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xms4G -Xmx4G -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /var/lib/jenkins/.gradle/wrapper/dists/gradle-6.0-all/cra2s42cvogxluqqpvbc5e9xd/gradle-6.0/lib/gradle-launcher-6.0.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 6.0
jenkins 23580 1 1 21:45 ? 00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xms4G -Xmx4G -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /var/lib/jenkins/.gradle/wrapper/dists/gradle-6.0-all/cra2s42cvogxluqqpvbc5e9xd/gradle-6.0/lib/gradle-launcher-6.0.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 6.0
jenkins 24052 1 1 21:46 ? 00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xms4G -Xmx4G -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /var/lib/jenkins/.gradle/wrapper/dists/gradle-6.0-all/cra2s42cvogxluqqpvbc5e9xd/gradle-6.0/lib/gradle-launcher-6.0.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 6.0
jenkins 24081 1 1 21:46 ? 00:00:06 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xms4G -Xmx4G -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /var/lib/jenkins/.gradle/wrapper/dists/gradle-6.0-all/cra2s42cvogxluqqpvbc5e9xd/gradle-6.0/lib/gradle-launcher-6.0.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 6.0
jenkins 26789 1 9 21:54 ? 00:00:05 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xms4G -Xmx4G -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /var/lib/jenkins/.gradle/wrapper/dists/gradle-6.0-all/cra2s42cvogxluqqpvbc5e9xd/gradle-6.0/lib/gradle-launcher-6.0.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 6.0
jenkins 26821 1 9 21:54 ? 00:00:04 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xms4G -Xmx4G -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /var/lib/jenkins/.gradle/wrapper/dists/gradle-6.0-all/cra2s42cvogxluqqpvbc5e9xd/gradle-6.0/lib/gradle-launcher-6.0.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 6.0
jenkins 26853 1 11 21:54 ? 00:00:05 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xms4G -Xmx4G -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /var/lib/jenkins/.gradle/wrapper/dists/gradle-6.0-all/cra2s42cvogxluqqpvbc5e9xd/gradle-6.0/lib/gradle-launcher-6.0.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 6.0
Adding --debug to the --no-daemon command looks like:
019-11-10T21:57:10.779-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.TcpOutgoingConnector] Trying to connect to address /0:0:0:0:0:0:0:1.
2019-11-10T21:57:10.784-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.TcpOutgoingConnector] Cannot connect to address /0:0:0:0:0:0:0:1, skipping.
2019-11-10T21:57:10.784-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.TcpOutgoingConnector] Trying to connect to address /127.0.0.1.
2019-11-10T21:57:10.785-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.TcpOutgoingConnector] Connected to address /127.0.0.1:38003.
2019-11-10T21:57:10.820-0800 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClient] Connected to daemon DaemonInfo{pid=28076, address=[1fadbe66-bb79-478f-afff-b43881850ffc port:38003, addresses:[/0:0:0:0:0:0:0:1, /127.0.0.1]], state=Busy, lastBusy=1573451830729, context=DefaultDaemonContext[uid=5c15a660-9b10-4cca-870a-4b7122057d2e,javaHome=/usr/lib/jvm/java-11-openjdk-amd64,daemonRegistryDir=/Local/jenkins/.gradle/daemon,pid=28076,idleTimeout=120000,priority=NORMAL,daemonOpts=–add-opens,java.base/java.util=ALL-UNNAMED,–add-opens,java.base/java.lang=ALL-UNNAMED,–add-opens,java.base/java.lang.invoke=ALL-UNNAMED,–add-opens,java.prefs/java.util.prefs=ALL-UNNAMED,-Xms4G,-Xmx4G,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]}. Dispatching request Build{id=28f3b28b-68e7-4cc5-bbc7-a7d4caad40b0, currentDir=/Local/jenkins/workspace/Synthesis}.
2019-11-10T21:57:10.821-0800 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientConnection] thread 1: dispatching class org.gradle.launcher.daemon.protocol.Build
2019-11-10T21:57:10.823-0800 [DEBUG] [org.gradle.internal.remote.internal.inet.SocketConnection] Discarding EOFException: java.io.EOFException
2019-11-10T21:57:10.834-0800 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
2019-11-10T21:57:10.834-0800 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] FAILURE: Build failed with an exception.
With the daemon, there’s no obvious exception or error that I can see.
Anyhow, rolling back to the previous version works – how can this be debugged further? Thanks.
Eric