Will this aspect of performance be resolved in 1.0

Old Forum Archive
1

I have a performance problem in milestone-3 that doesn’t seem to be resolved in the latest milestone-5 snapshot. I understand that there are many performance improvements left to go for on the journey to milestone-5, 1.0-rc, and 1.0.

I have a project with ALOT of dependencies – Over 150 (that includes the transitive dependencies). On top of that, I have 5 maven repositories where they are getting pulled from. It seem that with every build, gradle continues to make HTTP HEAD requests for both the .pom and .jar file for these repositories. – Even when these files exist in the cache.

This is taking a long time to iterate over these dependencies and repositories.

I understand that work has been added in the new Wharf Local cache to manage the integrity of the cache, so perhaps that is why it has to continue to hit the respositories each time, but if so, is this optional, or configurable via a switch?

I could understand you might want to factor in build integrity for CI builds or official release builds, but as a developer trying to rapidly iterate on development, these hits are really dragging things down.

Can someone comment on whether this aspect of performance will indeed be fixed by 1.0, or even earlier? I’d be happy to share more detail on my build if it would help.

Thanks.

Doug

2

Fixing performance when a build has multiple repositories defined is next up (we’ve already started on it), so you shouldn’t have to wait for 1.0. It should be in the release after 1.0-milestone-5.

3

Thanks, Adam. One more question if you don’t mind.

It seems the problem I’m describing isn’t so much related to multiple repositories, but the fact that gradle is even going to the repositories at all after a successful previous run of the build put all the needed artifacts into the local cache. It should have everything it needs already.

I could see the need in the case for snapshots where you might want to continue hitting the repos looking for updates, but these are fixed release versions.

As mentioned in my original post, I can see the need/value for checking the integrity of the cache for official CI/release builds, but as a developer, I have a need for speed, and hitting the repos each time is incredibly painful.

Especially in my case right now. I’m trying to convert a legacy ant build system to either maven or gradle. Honestly gradle is my preference right now, as I am a big fan of groovy, and have plenty of experience fighting maven at times, so I completely buy into gradle. However, it will take some political effort and evangalizing on my part to introduce YABT (Yet Another Build Tool) into our enterprise – Especially one not even 1.0 yet, so I’m taking the the time now to pilot my build migration with both maven and gradle, grain experience, and (hopefully) demonstrate the benefits of gradle over maven. That being said, if gradle doesn’t meet our needs, we would already have a head start with moving to maven.

I have a portion of the maven pom.xml working and I’m now trying to reach parity with a build.gradle equivalent. But where as my maven build runs the ‘clean test-compile’ goals in 19 seconds, running ‘gradle clean compileTestJava’ is taking 1 minute 36 seconds.

This is even more painfully slow right now as I’m trying right now to get my gradle test task working with testng, with the correct configuration. Every time I make a tweak, I have to wait a minute and a half just to get to the test task that I’m changing.

Will this be fixed, or will there be some option in 1.0 to prevent the gradle from hitting the repo each time the build is run if it has a fixed release in the cache?

Is this as simple as supplying an -offline switch on the command line?

Thanks again.

4

I second an offline switch. It should even allow snapshots to not be checked. Right now if you don’t have internet just compiling your code with existing snapshots can be impossible.

5

Yes, I consider an offline switch to be critical to 1.0. Looks like there is already a JIRA issue for this - http://issues.gradle.org/browse/GRADLE-320. I added my vote there.

Its funny. I’m about to board a plane in a few hours and was thinking of continuing to work and refine my gradle build, but it hadn’t even registered that I can’t do this if internet connectivity is required :frowning:

6

The problem as described shouldn’t be related to multiple repositories, but unfortunately due to the implementation of ivy ChainResolver, it is. What’s happening is that for each repository, we first look in the cache, then look for the remote artifact. So the only time you avoid the remote lookup is when the dependency is found in the cache of the first defined repository.

Obviously, this is not great! There is a fix for this in development; it won’t make it into milestone-5, but this problem will certainly be fixed for the following release.

Once we’ve optimised dependency resolution for static revisions, we’ll start looking at a consistent, performant solution to the problem of dynamic revisions. This will include the ability to use cached versions of dynamic revisions when running offline.

7

Thank you both Adam and Daz. I understand clearly the cause of the problem I’m running into and am glad it’s being addressed by 1.0.

Regarding support for an offline switch, from your answer it sounds like its next on the priority list after resolving the ivy chain resolver, but it wasn’t clear if the feature is a requirement to be be released in 1.0. I’m hoping so.

Thanks again.

Doug

8

For 1.0, we want to have some reasonable behaviour for when you’re offline. The solution may or may not include adding an offline switch. I’m hoping we can come up with a solution where Gradle can just do the right thing automatically, without the need for a switch. But if we can’t, then a switch makes a lot of sense.

9

Just to followup on this, now that I understand the issue currently is related to multiple repositories, I created a nexus group repository and aggregated the 5 repositories I had directly specified in my build.gradle file – something I now realize I probably should have one from the start. After updating my repo to point to that one consolidated url, my gradle builds went from 1 minute 36 seconds to 12 seconds, which come in line and even beats the 15 seconds it was taking with maven, so I am VERY happy. Thank you so much for your help… GO GRADLE!!!