Dependency resolution related heap utilisation issues on a large, multi-project build


(Danny Thomas) #1

I’m using Gradle 1.6 and I have a large, 174 project multi-project build which is throwing OutOfMemoryErrors when running tasks against all projects (‘idea’ in this case). I can reproduce with and without the daemon with a 1024m heap. Eclipse MAT’s dominator tree shows that ‘DefaultConfiguration_Decorated’ ‘cachedResolverResults’ instance is the where references are being held.

For example, a configuration for a simple project with a total of 106 dependencies reported by the ‘dependencies’ task for the main configuration consumes 1,707,304 bytes. ‘org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration’ holds references to ‘DefaultResolvedArtifact’ and ‘DefaultResolvedDependency’ where the utilisation is actually occurring. An artifact such as hibernate-core retains 95,152 bytes:

Class Name
                                                                                                                               | Shallow Heap | Retained Heap | Percentage
    ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    org.gradle.api.internal.artifacts.configurations.DefaultConfiguration_Decorated @ 0x129fadb50
                                            |
        144 |
   1,707,304 |
    0.18%
    |- org.gradle.api.internal.artifacts.ResolverResults @ 0x14c7d0a40
                                                                       |
         24 |
   1,702,768 |
    0.18%
    |
|- org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingArtifactDependencyResolver$ErrorHandlingResolvedConfiguration @ 0x14c7d0a28|
         24 |
   1,647,432 |
    0.17%
    |
|
'- org.gradle.api.internal.artifacts.ivyservice.SelfResolvingDependencyResolver$FilesAggregatingResolvedConfiguration @ 0x14c7d0a10
|
         24 |
   1,647,408 |
    0.17%
    |
|
   |- org.gradle.api.internal.artifacts.ivyservice.DefaultResolvedConfiguration @ 0x14c7d07c0
                                      |
         16 |
   1,646,824 |
    0.17%
    |
|
   |
'- org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration @ 0x14c7a0808
                                    |
         40 |
   1,646,808 |
    0.17%
    |
|
   |
   |- org.gradle.api.internal.artifacts.DefaultResolvedArtifact @ 0x14c7c4bd0
                                                |
         40 |
      95,152 |
    0.01%
    |
|
   |
   |
|- org.gradle.api.internal.artifacts.ivyservice.ResolvedArtifactFactory$1 @ 0x14c7c4bf8
                                |
         24 |
      94,792 |
    0.01%
    |
|
   |
   |
|
|- org.apache.ivy.core.module.descriptor.MDArtifact @ 0x14c7380b8
                                                   |
         48 |
      94,624 |
    0.01%
    |
|
   |
   |
|
|
|- org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor @ 0x14c735100
                                   |
        112 |
      94,288 |
    0.01%
    |
|
   |
   |
|
|
|
|- java.util.ArrayList @ 0x14c735170
                                                                          |
         24 |
      80,824 |
    0.01%
    ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

References on the anonymous class ‘DefaultArtifactFactory$1’ appear to be the root problem. I hadn’t come across anonymous class leaks before, but now I think about it some more: the anonymous class being non-static, won’t be eligible for garbage collection until it’s parent object is collected. It’s referencing finals in the method, so it’ll also hold references to those.

Seems the solution is to replace the ‘Factory’ anonymous class with a static inner class.

References on the anonymous inner class:

Type|Name
      |Value
    ----------------------------------------------------------------------------------------------------------------------------------------------------
    ref |this$0
    |org.gradle.api.internal.artifacts.ivyservice.ResolvedArtifactFactory @ 0x11f8f91b8
    ref |val$resolver|org.gradle.api.internal.artifacts.ivyservice.ivyresolve.LazyDependencyToModuleResolver$ErrorHandlingArtifactResolver @ 0x14c74c1a0
    ref |val$artifact|org.apache.ivy.core.module.descriptor.MDArtifact @ 0x14c7380b8
    ----------------------------------------------------------------------------------------------------------------------------------------------------

Code for reference. From GitHub today, releases branch:

public ResolvedArtifact create(ResolvedDependency owner, final Artifact artifact, final ArtifactResolver resolver) {
        return new DefaultResolvedArtifact(owner, artifact, new Factory<File>() {
            public File create() {
                return lockingManager.useCache(String.format("download %s", artifact), new Factory<File>() {
                    public File create() {
                        DefaultBuildableArtifactResolveResult result = new DefaultBuildableArtifactResolveResult();
                        resolver.resolve(artifact, result);
                        return result.getFile();
                    }
                });
            }
        });
    }

(Szczepan Faber) #2

Hey,

I’m just working on this issue. We know that resolved configuration uses too much memory and most of that is not collectable. This may become a problem for large builds. As a short term measure, you can increase the heap size. Long-term - wait for 1.8 :slight_smile:

Good catch with the anonymous inner class. I’ve found another instance of this kind of leak.

I’m experimenting with heap usage on ‘sf-mem’ branch - you could try it out to build your project. Let me know if you want to help out with diagnosing / debugging the issue - we can team up.

Cheers!


(Danny Thomas) #3

From what I could tell, this one reference is responsible for most of the overhead. Happy to help out - will check out that branch tomorrow and run it with YourKit.


(Szczepan Faber) #4

I’ll fix this anonymous instance in the branch and give it a spin. Btw. the project I work with has 3500 modules :wink:


(Szczepan Faber) #5

Actually, before making any further changes I’d like to know how the sf-mem branch Gradle worked for you. To use the locally built gradle, run ‘./gradlew install’ from gradle’s root dir (after cloning and checking out my branch).


(Danny Thomas) #6

3500 - that’s crazy. No worries, will check it out tomorrow and let you know.


(Danny Thomas) #7

I can’t run the same task with that branch - code has been commented out and replaced with UOEs:

Caused by: java.lang.UnsupportedOperationException: Forget it

at org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration.getFirstLevelModuleDependencies(DefaultLenientConfiguration.java:90)


(Szczepan Faber) #8

Is this your custom plugin that fails? Instead of getFristLevelModuleDependencies() you can use a newer model of the dependency tree: someConf.incoming.resolutionResult see more: http://www.gradle.org/docs/current/javadoc/org/gradle/api/artifacts/ResolvableDependencies.html#getResolutionResult()


(Danny Thomas) #9

The ‘idea’ plugin is failing, which is what I was using when I ran into this problem.

Caused by: java.lang.UnsupportedOperationException: Forget it
 at org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration.getFirstLevelModuleDependencies(DefaultLenientConfiguration.java:90)
 at org.gradle.plugins.ide.internal.JavadocAndSourcesDownloader.resolveDependencies(JavadocAndSourcesDownloader.groovy:72)
 at org.gradle.plugins.ide.internal.JavadocAndSourcesDownloader.this$2$resolveDependencies(JavadocAndSourcesDownloader.groovy)
 at org.gradle.plugins.ide.internal.JavadocAndSourcesDownloader$this$2$resolveDependencies.callCurrent(Unknown Source)
 at org.gradle.plugins.ide.internal.JavadocAndSourcesDownloader.<init>(JavadocAndSourcesDownloader.groovy:42)
 at org.gradle.plugins.ide.internal.IdeDependenciesExtractor.extractRepoFileDependencies(IdeDependenciesExtractor.groovy:73)
 at org.gradle.plugins.ide.internal.IdeDependenciesExtractor$extractRepoFileDependencies.call(Unknown Source)
 at org.gradle.plugins.ide.idea.model.internal.IdeaDependenciesProvider.getModuleLibraries(IdeaDependenciesProvider.groovy:70)
 at org.gradle.plugins.ide.idea.model.internal.IdeaDependenciesProvider$_provide_closure3.doCall(IdeaDependenciesProvider.groovy:45)
 at org.gradle.plugins.ide.idea.model.internal.IdeaDependenciesProvider.provide(IdeaDependenciesProvider.groovy:44)
 at org.gradle.plugins.ide.idea.model.internal.IdeaDependenciesProvider$provide.call(Unknown Source)
 at org.gradle.plugins.ide.idea.model.IdeaModule.resolveDependencies(IdeaModule.groovy:302)
 at org.gradle.plugins.ide.idea.model.IdeaModule.mergeXmlModule(IdeaModule.groovy:338)

(Szczepan Faber) #10

Oh, i see. My experiment is not yet compatible with the idea plugin.

The problem is that resolved dependency tree (the old model, the ResolvedDependency class and friends) is pretty expensive and it is strongly referenced from the Configuration object. The solution that I’m exploring now is getting rid of the old model, it will be deprecated in some fashion until removed in say Gradle 2.0.

Idea plugins still use the old model so I need to fix that, too.


(Szczepan Faber) #11

Can you try implementing your anonymous inner class fix to the master and trying out with your build? My gut feel says it won’t help but let’s see.


(Danny Thomas) #12

Even without any change it’s looking much better. I think this is already taken care of on master; I don’t see Ivy related references dangling from the configuration, like I do on 1.6. Looks like most of the overhead in the configuration is now the LinkedHashMap entries on DefaultResolvedDependency (children/parents/parentArtifacts) @ 40 bytes per entry:

https://code.google.com/p/memory-measurer/wiki/ElementCostInDataStructures

Not familiar enough with the project to comment on whether you could make savings there. parentArtifacts would be easier to handle if you used a Guava Multiset though.


(Szczepan Faber) #13

Yes, we’ve already done changes in 1.7 that improve memory footprint. Make sure you try out the release candidate!


(Danny Thomas) #14

Will do! Thanks.