Build stops with message: "Gradle build daemon has been stopped: service stop"

Szczepan_Faber · November 18, 2016, 8:38pm

At LinkedIn we observe intermittent daemon misbehavior with Gradle 3.1, daemon stops with message:

Gradle build daemon has been stopped: service stop

Above message looks different than when the daemon is externally stopped using ‘gradle --stop’.

More complete build log:

[INFO] Using versions:
[INFO] Environment: Gradle 3.1, JDK 1.8.0_40
[INFO] Gradle Plugins: client-typed-urn-generator:1.0.18,
ligradle-jvm:1.3.8, lix-cleanup-plugin:0.1.10, mntf-ios-plugins:1.0.0,
rum-ios:3.0.29, tracking-constants:0.1.4
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
MaxPermSize=1024m; support was removed in 8.0
Starting a Gradle Daemon, 3 incompatible and 1 stopped Daemons
could not be reused, use --status for details
FAILURE: Build failed with an exception.
What went wrong:
Gradle build daemon has been stopped: service stop
Try:
Run with --stacktrace option to get the stack trace. Run with
--info or --debug option to get more log output.
mint setup | [ERROR] './scripts/setup.sh' returned 1

Any pointers / suggestions?

eriwen · November 18, 2016, 9:18pm

@Szczepan_Faber Could you please send us the interesting tail-end of ~/.gradle/daemon/3.1/daemon-PID.out.log? That will help us determine the source of the problem.

deep00maj · November 18, 2016, 10:07pm

The forum is telling me new users cannot upload file. I will ask Szczepan to upload the file

Szczepan_Faber · November 19, 2016, 1:11am

Thanks for looking into this issue!!!

Here’s the relevant excerpt from the log. I pasted full log to the slack channel.

18:21:22.079 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'command '/usr/bin/vm_stat''. Working directory: /export/home/tester/.gradle/daemon/3.1 Command: /usr/bin/vm_stat 
18:21:22.079 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Environment for process 'command '/usr/bin/vm_stat'': {IRBRC=/export/home/tester/.rvm/rubies/ruby-2.1.0/.irbrc, PATH=/export/home/tester/mint/bin:/opt/CollabNet_Subversion/bin:/export/apps/xtools/bin:/opt/CollabNet_Subversion/bin:/export/apps/xtools/bin:/export/home/tester/.nvm/v0.10.21/bin:/export/home/tester/.rvm/gems/ruby-2.1.0/bin:/export/home/tester/.rvm/gems/ruby-2.1.0@global/bin:/export/home/tester/.rvm/rubies/ruby-2.1.0/bin:/usr/local/bin:/usr/local/sbin:/export/home/tester/adt-bundle-mac/sdk/tools:/export/home/tester/adt-bundle-mac/sdk/platform-tools:/export/home/tester/.rvm/bin:/usr/local/linkedin/bin:/export/content/linkedin/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/linkedin/bin:/export/apps/apache-maven/bin:/usr/local/linkedin/bin:/export/apps/apache-maven/bin:/export/content/linkedin/bin, EXECUTION_TYPE=pcl, BUILD_CATEGORY=PROD, HUDSON_HOME=/export/home/tester/hudson//data, VARIANT_XCODE=7.3, SECONDARY=True, ENABLE_CIA=true, MP_TRUNKDEV_VERSION=0.1.184, ANDROID_HOME=/export/home/tester/adt-bundle-mac/sdk, MINT_VERSION=4.1.117, LC_PAPER=en_US.UTF-8, rvm_version=1.26.9 (1.26.9), PWD=/export/home/tester/hudson/data/workspace/MP_TRUNKDEV_POSTCOMMIT/voyager-ios_2c9e7782e9f96e8cea77485929a9fc8f4c429b81, HUDSON_BIN_REVISION=146096, NVM_PATH=/export/home/tester/.nvm/v0.10.21/lib/node, HUDSON_SLAVE_DST=/export/home/tester/hudson/, MY_RUBY_HOME=/export/home/tester/.rvm/rubies/ruby-2.1.0, JAVA_OPTIONS_SLAVE=-Xms4G -Xmx4G -XX:-MaxFDLimit -XX:MaxPermSize=512m -Dhudson.util.ProcessTreeKiller.disable=true -Dhudson.remoting.Launcher.pingTimeoutSec=1000 -Dhudson.remoting.Launcher.pingIntervalSec=5000 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=12942 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false, IS_PROD_MPTRUNKDEV=true, rvm_path=/export/home/tester/.rvm, NODE_NAME=lca1-tools0430, APP_ICON_47162=/export/home/tester/.gradle/ligradle/gradle-3.1/media/gradle.icns, BOUNCES=3, POOL=POOL-MP-ELCAPITAN, TIMEOUT=240, LC_CTYPE=en_US.UTF-8, PROD_SPEC_VERSION=9.7.1228, SHLVL=5, KICK_OFF_DEPENDENCY_JOBS=True, UPDATE_TM=true, LEOHOME=/export/content/network, VARIANT_INDEX=2, JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home, TM_URL=http://localhost/disabled, MINT_WROTE_DEPENDENCY_SPEC=TRUE, HUDSON_URL=https://lva1-tools1999.corp.linkedin.com/, VARIANT_R_OS=darwin, WORKSPACE=/export/home/tester/hudson/data/workspace/MP_TRUNKDEV_POSTCOMMIT, LC_NAME=en_US.UTF-8, SSH_CLIENT=10.136.248.29 44187 22, HUDSON_TRACKING_CRUMB=e2d8c037-a002-414b-81c3-10d0da9a91e3, PCS_ONLY=False, NLS_LANG=American_America.UTF8, PARENT_EXECUTION_URL=https://lva1-tools1999.corp.linkedin.com/job/MP_TRUNKDEV_POSTCOMMIT/4805/, GEM_PATH=/export/home/tester/.rvm/gems/ruby-2.1.0:/export/home/tester/.rvm/gems/ruby-2.1.0@global, SSH_CONNECTION=10.136.248.29 44187 10.252.177.140 22, BUILD_NUMBER=4813, BUILD_URL=https://lva1-tools1999.corp.linkedin.com/job/MP_TRUNKDEV_POSTCOMMIT/4813/, NVM_BIN=/export/home/tester/.nvm/v0.10.21/bin, LC_MEASUREMENT=en_US.UTF-8, EXECUTION_UUID=86b829ae-ad35-11e6-be5f-00200afcb18c, LC_TELEPHONE=en_US.UTF-8, GRADLE_OPTS=-Xms512m -Xmx2500m -XX:PermSize=256m -XX:MaxPermSize=1024m "-Xdock:name=Gradle" "-Xdock:icon=/export/home/tester/.gradle/ligradle/gradle-3.1/media/gradle.icns", TRUNKDEV_MP_REPO_REVISION=1487689, LC_TIME=en_US.UTF-8, JOB_URL=https://lva1-tools1999.corp.linkedin.com/job/MP_TRUNKDEV_POSTCOMMIT/, LC_MESSAGES=en_US.UTF-8, HUDSON_COOKIE=6a21016d-a8c0-4abb-a4e5-3ac56b23bf46, DISTRIBUTED_TEST_NUMBER_CAP=12, MAIL=/var/mail/tester, LC_COLLATE=en_US.UTF-8, JAVA_MAIN_CLASS_47162=org.gradle.launcher.GradleMain, LOGNAME=tester, PUBLISH_WAR_MANIFEST=true, SHELL=/bin/bash, BUILD_TAG=hudson-MP_TRUNKDEV_POSTCOMMIT-4813, LC_ADDRESS=en_US.UTF-8, REQUESTS_CA_BUNDLE=/usr/local/linkedin/lib/python2.6/site-packages/linkedin/ssl/ca-bundle.crt, OLDPWD=/export/home/tester/.gradle/ligradle/gradle-3.1, JAVA_MAIN_CLASS_47169=org.gradle.launcher.daemon.bootstrap.GradleDaemon, PRODUCT_NAME=voyager-ios, GEM_HOME=/export/home/tester/.rvm/gems/ruby-2.1.0, VERSIONER_PYTHON_PREFER_32_BIT=no, TMPDIR=/export/home/tester/TEMP, RUBY_VERSION=ruby-2.1.0, _system_name=OSX, NVM_IOJS_ORG_MIRROR=https://iojs.org/dist, __CF_USER_TEXT_ENCODING=0x1F7:0:0, CIA_ENV=prod, LC_IDENTIFICATION=en_US.UTF-8, BUILD_USER=khu, LC_MONETARY=en_US.UTF-8, MANPATH=/export/home/tester/.nvm/v0.10.21/share/man:/usr/local/share/man:/export/home/tester/.rvm/man:/usr/local/linkedin/man:/usr/share/man:/Applications/Xcode-7.3.app/Contents/Developer/usr/share/man:/Applications/Xcode-7.3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/share/man, _system_arch=x86_64, rvm_bin_path=/export/home/tester/.rvm/bin, rvm_prefix=/export/home/tester, BUILD_ID=2016-11-17_18-19-36, LANG=en_US.UTF-8, JOB_NAME=MP_TRUNKDEV_POSTCOMMIT, VARIANT_FLAVOR=debug, _MP_TRUNKDEV_OVERRIDE_BINARY_CACHE_PROXY=corp-lva1, APP_NAME_47162=Gradle, VERSIONER_PYTHON_VERSION=2.6, NODE_LABELS=POOL-MP-ELCAPITAN POOL-MP-ELCAPITAN-LCA1 lca1-tools0430, USE_MP_TRUNKDEV_MP=true, SCM_REVISION=2c9e7782e9f96e8cea77485929a9fc8f4c429b81, NVM_NODEJS_ORG_MIRROR=https://nodejs.org/dist, NVM_DIR=/export/home/tester/.nvm, JAVA_MAIN_CLASS_10365=hudson.remoting.Launcher, HUDSON_SERVER=lva1-tools1999.corp.linkedin.com, _system_type=Darwin, EXECUTOR_NUMBER=0, HUDSON_SERVER_COOKIE=226406890342ebbdae7043056c65a3a9, _system_version=10.11, USER=tester, LC_NUMERIC=en_US.UTF-8, HUDSON_HOME_SCRIPTS=/export/content/lid/data/hudson-i001/hudson-scripts, BRANCH_NAME=master, HOME=/export/home/tester}
18:21:22.080 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: STARTING
18:21:22.081 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 15: received class org.gradle.launcher.daemon.protocol.CloseInput
18:21:22.081 [DEBUG] [org.gradle.launcher.daemon.server.DefaultDaemonConnection] thread 15: Received IO message from client: org.gradle.launcher.daemon.protocol.CloseInput@6c8b1f73
18:21:22.082 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Waiting until process started: command '/usr/bin/vm_stat'.
18:21:22.084 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: FAILED
18:21:22.084 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Process 'command '/usr/bin/vm_stat'' finished with exit value -1 (state: FAILED)
18:21:22.085 [DEBUG] [org.gradle.launcher.daemon.server.Daemon] stop() called on daemon
18:21:22.085 [INFO] [org.gradle.launcher.daemon.server.Daemon] Stop requested. Daemon is removing its presence from the registry...

eriwen · November 19, 2016, 2:40am

This is captured as https://github.com/gradle/gradle/issues/897 and we have a pretty good idea what the cause is. We’ll make sure this is fixed in 3.3.

Sorry for the trouble and thanks for reporting!