Out of memory in eclipse and idea plugin


(Corneil du Plessis) #1

We have a large project with many modules. The project builds fine but when we try to import into eclipse we get out of memory exceptions. Executing on command line with GRADLE_OPTS="-Xmx2048m -XX:MaxPermSize=256m -XX:-UseGCOverheadLimit -XX:+HeapDumpOnOutOfMemoryError -XX:+UseParallelGC -XX:+UseParallelOldGC" Last message is ‘Resolving dependencies’ on testRuntime of one of the modules.

It is a different module depending on the size of -Xmx The .hprof file is the size of the heap at the time of the dump so it isn’t feasible to send it.

  Class Name

































































 | Shallow Heap | Retained Heap | Percentage  ----------------------------------------------------------------------------------------------------------------------------------------------------------  java.lang.Thread @ 0x737930d8  main Thread











































  |





 104 |

 45 850 520 |



4,61%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xadf69c88





















|





 112 |

  2 574 392 |



0,26%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xae441008





















|





 112 |

  2 574 392 |



0,26%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xabff1280





















|





 112 |

  2 558 784 |



0,26%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xac737c68





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xacbffbe0





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xad35fdd8





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xad8291a0





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xadaa0d78





















|





 112 |

  2 431 456 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xae1e1140





















|





 112 |

  2 417 872 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xae6c1b60





















|





 112 |

  2 416 864 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xae9234e8





















|





 112 |

  2 416 864 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xad0c88a0





















|





 112 |

  2 415 880 |



0,24%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xace76ed0





















|





 112 |

  2 274 536 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xad5d73d0





















|





 112 |

  2 274 536 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xac2961d0





















|





 112 |

  2 274 512 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xac4e6cb0





















|





 112 |

  2 274 512 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xac9aefb0





















|





 112 |

  2 274 512 |



0,23%  |- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0xadd18f38





















|





 112 |

  2 274 512 |



0,23%  |- java.lang.ThreadLocal$ThreadLocalMap @ 0x73793170





































 |





  24 |



  56 568 |



0,01%  |- org.gradle.cli.CommandLineParser @ 0x737dea78







































  |





  24 |



  28 880 |



0,00%  |- groovy.lang.Reference @ 0xa8768938















































 |





  16 |



  22 464 |



0,00%  |- org.gradle.cli.CommandLineParser @ 0x737908e8







































  |





  24 |



  11 232 |



0,00%  |- org.apache.ivy.Ivy @ 0x81379bc8

















































 |





  64 |



  11 064 |



0,00%  |- org.gradle.api.internal.DefaultClassPathRegistry @ 0x73790950





























 |





  16 |



  10 904 |



0,00%  |- org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder$ResolveState @ 0xac289df0|





  48 |





6 696 |



0,00%  '- Total: 25 of 3 689 entries; 3 664 more













































|







  |









|







 org.gradle.api.internal.artifacts.configurations.DefaultConfiguration_Decorated @ 0x746aefb0











|





 144 |

 27 352 344 |



2,75%  class sun.awt.AppContext @ 0x64051be8 System Class







































|





  32 |

  4 184 920 |



0,42%  org.gradle.api.internal.artifacts.DefaultResolvedArtifact @ 0x951168d0

























 |





  40 |

  2 713 864 |



0,27%  ----------------------------------------------------------------------------------------------------------------------------------------------------------  

(Peter Niederwieser) #2

‘GRADLE_OPTS’ only affects the client JVM, not the Gradle Daemon. You need to set the ‘org.gradle.jvmargs’ property in ‘gradle.properties’ (per-project) or ‘~/.gradle/gradle.properties’ (per-user). As far as I know, the daemon uses 1G max heap by default.


(Corneil du Plessis) #3

From what I have seen the GRADLE_OPTS are used when creating a daemon. I tried same options in gradle.properties but still got out of memory errors.

The memory usage using eclipse and idea tasks seems to be a problem because there is not way an extra GB of ram is not enough.


(Peter Niederwieser) #4

From what I have seen the GRADLE_OPTS are used when creating a daemon.

How do you tell?

The memory usage using eclipse and idea tasks seems to be a problem because there is not way an extra GB of ram is not enough.

Which task(s) are you running exactly? Memory usage of IDE tasks should be dominated by dependency resolution. For very large builds, 1 or 2 GB may not be enough.


(Corneil du Plessis) #5

From what I have seen the GRADLE_OPTS are used when creating a daemon. > How do you tell? ps -ef | grep java | grep gradle

I can see the options passed to daemon are the same as GRADLE_OPTS

The memory usage using eclipse and idea tasks seems to be a problem because there is not way an extra GB of ram is not enough. > Which task(s) are you running exactly? Memory usage of IDE tasks should be dominated by dependency resolution. For very large builds, 1 or 2 GB may not be enough.

./gradlew idea OR ./gradlew eclipse

Even 180 modules with 10-20 dependencies each should not struggle with 2GB of ram. The dependencies or not even unique per module there is a lot of overlap with total number of external dependencies about 50-80. If it will help to know that most of the external dependencies are ivy and not maven dependencies.


(Peter Niederwieser) #6

I can see the options passed to daemon are the same as GRADLE_OPTS.

I cannot reproduce this.


(Corneil du Plessis) #7

A project with 100+ sub projects is probably what is needed.


(Peter Niederwieser) #8

That won’t make any difference in terms of GRADLE_OPTS (not) effecting the daemon.


(Radim Kubacki) #9

If 45850520 bytes retained by main thread is 4,61% of heap size than it is not 2GB large IMO. BTW: is it heap size OOME or any other kind of out-of-memory error.

Of course I agree that idea and eclipse tasks should not be so eager in their memory consumption but I think we need to come to an agreement how much memory it actually uses and how to affect that.