Multiple instances and high memory consumption by gradle daemon

Bugs
1

Hello,

I am using the daemon with gradlew (version 2.5) when building a simple Hello World application. However, once the daemon is started it keeps consuming a lot of memory. In order to build it launches 15 instances of the daemon which all persist after the build has finished. Memory consumption also grows with every build/run. Initially, the daemon took ca. 140 mB of RAM, after a few builds it is up to 177 mB.
On larger applications the memory consumption is even worse and will eventually crash the system.

My system specs:
OS: Debian Jessie, 64-bit, Kernel 3.16
Java: Oracle Java, 1.8
Gradle: Version 2.

2

That seems strange that you see 15 different processes. Does the problem exist with Gradle 2.9? Can you explain steps how to reproduce the problem? Can you reproduce the problem in another environment?

This might help reducing memory usage: Continuous mode eating too much memory .

3

Thanks for taking a look.
The problem also exists with gradle 2.9.
I am still quite new to gradle but as far as I can tell I am not using continuous build. If I understand correctly it is not activated by default and I did not take any steps to do so.
At the moment, I do not have the possibility to test on a different OS. I might be able to do so over the weekend.

Since I am new to this forum I cannot upload any files. So here is my project setup:

hello_world_root/
|__hello_world_main/
|   |__src/
|   |   |__main/
|   |       |__java/
|   |           |__com/
|   |               |__example/
|   |                   |__hello/
|   |                       |__Main.java
|   |__build.gradle
|__build.gradle
|__settings.gradle

The file contents are:

$ cat hello_world_root/build.gradle 
apply plugin: 'eclipse'

task wrapper(type: Wrapper) {
	gradleVersion = '2.9'
}

$ cat hello_world_root/settings.gradle 
include ':hello_world_main'

$ cat hello_world_root/hello_world_main/build.gradle 
apply plugin: 'java'
apply plugin: 'application'

mainClassName = "com.example.hello.Main"

sourceCompatibility = 1.7
targetCompatibility = 1.7

repositories {
}

task wrapper(type: Wrapper) {
	gradleVersion = '2.9'
}

$ cat hello_world_root/hello_world_main/src/main/java/com/example/hello/Main.java 
package com.example.hello;

public class Main {
public static void main(String[] args) {
	System.out.println("Hello World!");
}
}

In order to reproduce I issue the following commands in a Linux terminal:
$ gradle wrapper # this runs my system's gradle (version 1.5) $ pkill java # kill all JVMs to make sure no daemon from version 1.5 is lurking around
After that I issue the following three commands a couple of times.
$ ./gradlew clean $ ./gradlew build $ ./gradlew run
Memory usage increases after each cycle. When I use $ ps -eT I can see that there are appr. 15 idle daemon threads, all running inside the same JVM. The number of thread varies; sometimes there are fewer other times there are more. However, what struck me as odd is the memory consumption. Over 140 MB (initially) just to build and run the application seems excessive and it increases with each build.

4

Thanks for the good explanation and instructions about how to reproduce it.

I think it’s ok when you see only a single daemon process with 15 threads and not 15 daemon processes. The JVM itself has certain threads running. When the daemon is idle, I see 24 threads in the jstack output which of 5 are threads created by Gradle. That’s with Oracle Java 1.8.0_66 on Linux.

Gradle isn’t currently optimized to extremely low memory consumption. It does consume a lot of memory compared to some non-Java based tools.

You might want to tune the JVM memory settings if you want to optimize for low memory consumption: Continuous mode eating too much memory .

There is also information about the Gradle Daemon in the manual:
https://docs.gradle.org/current/userguide/gradle_daemon.html
That might answer some of your questions about it.

Do you have a particular reason to optimize for low memory use?

5

There have been major fixes to memory consumption in Gradle 2.9. Could you upgrade to Gradle 2.9? How does that build behave on Gradle 2.9?
There is a more recent fix in Gradle 2.10-rc-2 (already in 2.10-rc-1) related to memory use on Java 6 (< Java7) and zipTree/tarTree usage.

6

I tried gradle 2.9 and the memory consumption is similar. For an Android project I am currently working on memory usage is around 650 - 700 MB. It consists of only 5 subprojects and the resulting apk-file is only around 15 MB; graphics and sound included.
I tried to influence the behaviour by setting
$ export GRADLE_OPTS="-Xmx384m -Dorg.gradle.jvmargs='-Xmx256m -XX:MaxPermSize=64m'"
but I do not fully understand what this is supposed to achieve. Why are there two -XmX options? I thought that memory consumption should not exceed 256 MB. I can verify via ‘htop’ that the JVM is in fact called with those parameters (Xmx256m, MaxPermSize=64m). However, the JVM seems to ignore those settings and memory usage is at 530 MB.

Yes, I have limited memory, 2GB. I am also using Eclipse* for coding and Firefox for quick websearches. Both can also be resource hungry. If I do not keep an eye on memory consumption and occasionally restart one of the programs then my System freezes eventually. I have been using Firefox and Eclipse for some time now and I know that they can have a quite large memory footprint. I did not know that Gradle can be this resource hungry, too.

*Note: I am using Eclipse only for its Code Completion and Refactoring features. It is not in any way involved in the building process. The building is done in CLI only. Even the gradle specific files are edited exclusively via vim in CLI.

7

The first setting is for the Gradle cli JVM and the org.gradle.jvmargs setting is used for the Gradle Daemon. You can also specify this setting in a gradle.properties file. This should be higher that the value for the Gradle cli when daemon is used. I’d assume that you could set the cli to a pretty low value (64M ?) when the daemon is used. However this is not supported or tested.

Yes it is relatively resource hungry. I assume that optimizing for low memory usage won’t be high priority in Gradle core development.

Upgrading hardware (or just adding more memory) might be an easier way to workaround this. It’s usually worth upgrading a developer laptop memory to the full amount that the hardware supports. Some older laptops don’t support more than 2GB, but I’d assume that most support at least 8GB. I’ve usually used Kingston’s memory configurator to find the correct memory type and config for a specific laptop model. The cost of 8GB is less than $50 for a lot of laptop models.

I hope this helps. :slight_smile:

8

Thanks for clarifying.

Ok, I still have a desktop where I could run it comfortably. I just got used to my laptop and had no problems with memory so far. I was always able to run eclipse, firefox and even the android emulator simultaneously. My main concern was that gradle has a memroy leak or I might have it misconfigured when memory was exhausted by it.

It definitely does does. Memory issue aside, I have been impressed by gradle so far. Thanks for the support and keep up the good work.