Out of memory in eclipse and idea plugin

corneil · May 16, 2014, 1:42pm

We have a large project with many modules. The project builds fine but when we try to import into eclipse we get out of memory exceptions. Executing on command line with GRADLE_OPTS="-Xmx2048m -XX:MaxPermSize=256m -XX:-UseGCOverheadLimit -XX:+HeapDumpOnOutOfMemoryError -XX:+UseParallelGC -XX:+UseParallelOldGC" Last message is ‘Resolving dependencies’ on testRuntime of one of the modules.

It is a different module depending on the size of -Xmx The .hprof file is the size of the heap at the time of the dump so it isn’t feasible to send it.

  Class Name

































































 | Shallow Heap | Retained Heap | Percentage  ----------------------------------------------------------------------------------------------------------------------------------------------------------  java.lang.Thread @ 0x737930d8  main Thread











































  |





 104 |

 45 850 520 |



4,61%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xadf69c88





















|





 112 |

  2 574 392 |



0,26%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xae441008





















|





 112 |

  2 574 392 |



0,26%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xabff1280





















|





 112 |

  2 558 784 |



0,26%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xac737c68





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xacbffbe0





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xad35fdd8





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xad8291a0





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xadaa0d78





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xae1e1140





















|





 112 |

  2 417 872 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xae6c1b60





















|





 112 |

  2 416 864 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xae9234e8





















|





 112 |

  2 416 864 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xad0c88a0





















|





 112 |

  2 415 880 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xace76ed0





















|





 112 |

  2 274 536 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xad5d73d0





















|





 112 |

  2 274 536 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xac2961d0





















|





 112 |

  2 274 512 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xac4e6cb0





















|





 112 |

  2 274 512 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xac9aefb0





















|





 112 |

  2 274 512 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xadd18f38





















|





 112 |

  2 274 512 |



0,23%  |- java.lang.ThreadLocal$ThreadLocalMap @ 0x73793170





































 |





  24 |



  56 568 |



0,01%  |- org.gradle.cli.CommandLineParser @ 0x737dea78







































  |





  24 |



  28 880 |



0,00%  |- groovy.lang.Reference @ 0xa8768938















































 |





  16 |



  22 464 |



0,00%  |- org.gradle.cli.CommandLineParser @ 0x737908e8







































  |





  24 |



  11 232 |



0,00%  |- org.apache.ivy.Ivy @ 0x81379bc8

















































 |





  64 |



  11 064 |



0,00%  |- org.gradle.api.internal.DefaultClassPathRegistry @ 0x73790950





























 |





  16 |



  10 904 |



0,00%  |- org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder$ResolveState @ 0xac289df0|





  48 |





6 696 |



0,00%  '- Total: 25 of 3 689 entries; 3 664 more













































|







  |









|







 org.gradle.api.internal.artifacts.configurations.DefaultConfiguration_Decorated @ 0x746aefb0











|





 144 |

 27 352 344 |



2,75%  class sun.awt.AppContext @ 0x64051be8 System Class







































|





  32 |

  4 184 920 |



0,42%  org.gradle.api.internal.artifacts.DefaultResolvedArtifact @ 0x951168d0

























 |





  40 |

  2 713 864 |



0,27%  ----------------------------------------------------------------------------------------------------------------------------------------------------------

Peter_Niederwieser · May 16, 2014, 2:15pm

‘GRADLE_OPTS’ only affects the client JVM, not the Gradle Daemon. You need to set the ‘org.gradle.jvmargs’ property in ‘gradle.properties’ (per-project) or ‘~/.gradle/gradle.properties’ (per-user). As far as I know, the daemon uses 1G max heap by default.

corneil · May 19, 2014, 9:10am

From what I have seen the GRADLE_OPTS are used when creating a daemon. I tried same options in gradle.properties but still got out of memory errors.

The memory usage using eclipse and idea tasks seems to be a problem because there is not way an extra GB of ram is not enough.

Peter_Niederwieser · May 19, 2014, 9:44am

From what I have seen the GRADLE_OPTS are used when creating a daemon.

How do you tell?

The memory usage using eclipse and idea tasks seems to be a problem because there is not way an extra GB of ram is not enough.

Which task(s) are you running exactly? Memory usage of IDE tasks should be dominated by dependency resolution. For very large builds, 1 or 2 GB may not be enough.

corneil · May 19, 2014, 1:54pm

From what I have seen the GRADLE_OPTS are used when creating a daemon. > How do you tell? ps -ef | grep java | grep gradle

I can see the options passed to daemon are the same as GRADLE_OPTS

The memory usage using eclipse and idea tasks seems to be a problem because there is not way an extra GB of ram is not enough. > Which task(s) are you running exactly? Memory usage of IDE tasks should be dominated by dependency resolution. For very large builds, 1 or 2 GB may not be enough.

./gradlew idea OR ./gradlew eclipse

Even 180 modules with 10-20 dependencies each should not struggle with 2GB of ram. The dependencies or not even unique per module there is a lot of overlap with total number of external dependencies about 50-80. If it will help to know that most of the external dependencies are ivy and not maven dependencies.

Peter_Niederwieser · May 19, 2014, 2:07pm

I can see the options passed to daemon are the same as GRADLE_OPTS.

I cannot reproduce this.

corneil · May 19, 2014, 2:46pm

A project with 100+ sub projects is probably what is needed.

Peter_Niederwieser · May 19, 2014, 3:00pm

That won’t make any difference in terms of GRADLE_OPTS (not) effecting the daemon.

Radim_Kubacki · May 19, 2014, 5:37pm

If 45850520 bytes retained by main thread is 4,61% of heap size than it is not 2GB large IMO. BTW: is it heap size OOME or any other kind of out-of-memory error.

Of course I agree that idea and eclipse tasks should not be so eager in their memory consumption but I think we need to come to an agreement how much memory it actually uses and how to affect that.