Large project with many modules runs out of memory compiling or resolving dependencies

Old Forum Archive
1

With GRADLE_OPTS=-Xmx2048m -XX:MaxPermSize=256m -XX:-UseGCOverheadLimit -XX:+HeapDumpOnOutOfMemoryError you should easliy build some of the largest projects ever devised. I have analyzed some dumps and notice org.gradle.api.internal.artifacts.configurations.DefaultConfiguration_Decorated is the ‘top dominator’ I someone would like to look at some reports please let me know

2

Can you give the exact error message? Is it running out of heap or permgen? We do know that configurations can consume a lot of memory.

3

java.lang.OutOfMemoryError: Java heap space

Usually while ‘Resolving dependencies’

4

HI Corneil,

There are some improvements in 1.7 regarding heap usage, and more planned in 1.8 - particularly WRT dependency management.

Is there any chance you could provide a sample build that uses a lot of memory like your build?

5

Our build has about 8000 files over 300 projects. I will not be able to provide you with the source.

6

Can you provide a yourkit memory analysis?

7

The memory usage problem went away when we resolved some issues related to dependencies. The project works by generating build.gradle files from a large number of ivy.xml files that was used previously by a set of ant build scripts. There was problems with the dependency resolution with the ant build which led to the switch to gradle. Once we fixed a few of the nasty transient dependencies the whole thing became more managable.

8

1.7 definitely improves things. You’re probably seeing the problem I reported two days ago:

http://forums.gradle.org/gradle/topics/dependency_resolution_related_heap_utilisation_issues_on_a_large_multi_project_build