Hi Guys,
Just tried out the new milestone-6 which Luke Daley pointed me to on Twitter. For at least one of our projects it keeps on going with the dependency resolution. It seems like it is continuously looping in the DependencyGraphBuilder. It might be that the dependencies we have together make up a cykel, which then makes it go round in circles.
I have a 3.1M gzipped log for the interested on dropbox: http://dl.dropbox.com/u/2619303/gradle-DependencyGraphBuilder.tar.gz
Regards, Jeroen
Thanks for the report Jeroen. Will take a look at the log and see if we can work out whats going on.
For the project in question, does ‘gradle dependencies’ complete?
Thx Luke!
gradle dependencies does not finish in a reasonable amount of time, ie I killed it after 10+ minutes. Both that and the build seem to be running insane once its doing the ‘Resolving dependencies’.
I’m having the same problem. Seems to be limited to my testCompile and testRuntime dependencies for a specific project.
I’m using a special ‘testFixtures’ configuration that extends from ‘testRuntime’ in order to accomplish project dependencies for unit test code in a multi-project build. So I define this:
configurations {
testFixtures {
extendsFrom testRuntime
}
}
In order to do something like this:
testCompile project(path: ":coreProject", configuration: "testFixtures")
Not sure if this is related or not but seems relevant.
For my setup, it seems there is a lot of conflict resolution taking place between different versions of modules that are direct dependencies vs transitive dependencies.
1.0-milestone-3 seems to handle it fine, but 1.0-milestone-6 does not. Haven’t tested other releases.
I have also posted zipped log files here: https://github.com/javaSpock/problem-gradle-1m6-dep-res/raw/master/gradle-logs.zip
Zip contains 2 log files, one from milestone-3 and one from milestone-6. In both cases I simply executed ‘gradle :ws:dependencies -d’, where ‘ws’ is obviously the name of one project in a multi-project build.
Tested with milestone-5 and had no problem. So it’s definitely an issue introduced with milestone-6.
I’m seeing the same issue with milestone-6.
With milestone-5 my build takes around 3 minutes but with milestone-6 it takes over 1 hour and 5 mins. In order for my build to succeed with milestone-6 I need to increase the heap to 1536m.
I have profiles for both milestone-5 and milestone-6 available but I need to scrub the client information out of them if anyone would like to see them.
Is it possible for you to post the output of ‘gradle dependencies’ somewhere?
I did a ‘gradle dependencies’ on the root project using milestone-6 and produced the following file containing the output:
I also want to add that all of the extra time for milestone-6 comes from dependency resolution
m6
All dependencies 1h2m34.03s
m5
All dependencies 21.924s
Below is a link to sample output from the m6 build, this type of logging goes on for nearly an hour (only the first 10000 lines are in the file).
I’ve been experiencing that same thing with 1.0m6.
I have a project with extensive dependency hierarchy that runs fine on v.0.9.2 but grinds to a halt on 1.0m6. (speed of 1.0m6 on smaller projects is impressive though) After few iterations of cleaning the cache between runs it seems to kind of slow down usually around same place (observing the download log statements).
Is there some good way of being able to examine what dependencies are giving m6 this grief? Not so sure that a dependency report would report to me which dependencies seem to problematic.
I’m going through a local nexus instance that has all necessary external maven repos proxied.
It’s unlikely to be a particular dependency. It’s more a result of having a deep dependency graph.
There should be a fix available soon. However, the fix probably won’t make it into milestone-7, but will certainly be available in milestone-8.
Thanks for the info Adam.
You all are doing a great job with gradle. I’ll wait patiently for the fix.
Could you be more explicit on what you think the problem is (or maybe I can find it on some other thread), if I wanted to take a look myself (remote debug for example)?
The basic problem is that the new algorithm considers every path through the dependency graph. This is because each edge in the graph can introduce excludes that affect the result deeper in the graph, and also to give accurate results when a conflict includes/excludes a particular subgraph. This approach doesn’t scale as the graph gets deeper.
So, we will tweak it to merge the paths. In particular, if there is a single path to a module version that does not exclude anything, we can ignore all the other incoming paths to that module version. And when this isn’t the case, we can use the intersection of the exclude rules for all incoming paths and have a single outgoing path (these two cases are really the same thing).
Adam, thanks for the insight.
You seem to have this in a firm grip, nice to know.
I’ll hold my horses until you’ve released the tweak.
Is there any jira issue I could follow on this?
Thanks again.
We’ve already linked the jira to this forum post. So since you are following this post (because you have participated in it), you’ll be notified when the issue is closed.
Verified the fix with 1.0-milestone-7-20111219065105+0100.
The same dependency graph which could not be resolved in milestone-6 was no problem for the milestone-7 snapshot. Performance seems to have drastically improved as well.
Thanks Adam and Gradle team! You’ve really built an outstanding tool. Keep up the good work.
That did the trick, my build now takes 2m 36s with the m7 snapshot.
First look is very promising. A build that is taking ages in 1.0m6 (>15 mins) is taking 5 mins in 1.0m7. Great work (and impressingly fast :)), thanks.
Will continue investigating with 1.0m7 and report if I hit any problems. Happy holidays!
Solved for me with m7. All of our builds are fine again!