Hey,
My build is taking a huge amount of time. The problem is that gradle is stuck on dependency resolution. The same build with maven runs perfectly and quickly.
The thing is that sometimes gradle is stuck 3-4 minutes because it can’t find a jar on a repo :
11:21:48.601 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Sending request: GET http://owasp-java-html-sanitizer.googlecode.com/svn/maven/net/sf/ehcache/ehcache-ee-root/2.7.0/ehcache-ee-root-2.7.0.pom HTTP/1.1
11:21:48.601 [DEBUG] [org.apache.http.headers] >> GET http://owasp-java-html-sanitizer.googlecode.com/svn/maven/net/sf/ehcache/ehcache-ee-root/2.7.0/ehcache-ee-root-2.7.0.pom HTTP/1.1
11:21:48.601 [DEBUG] [org.apache.http.headers] >> Accept-Encoding: gzip,deflate
11:21:48.601 [DEBUG] [org.apache.http.headers] >> Host: owasp-java-html-sanitizer.googlecode.com
11:21:48.601 [DEBUG] [org.apache.http.headers] >> Proxy-Connection: Keep-Alive
11:21:48.601 [DEBUG] [org.apache.http.headers] >> User-Agent: Gradle/1.6 (Mac OS X;10.7.5;x86_64) (Oracle Corporation;1.7.0_21;23.21-b01)
11:25:43.536 [DEBUG] [org.apache.http.impl.conn.DefaultHttpResponseParser] Garbage in response: 0
11:25:43.536 [DEBUG] [org.apache.http.impl.conn.DefaultHttpResponseParser] Garbage in response:
11:25:44.213 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Receiving response: HTTP/1.1 404 Not Found
11:25:44.213 [DEBUG] [org.apache.http.headers] << HTTP/1.1 404 Not Found
11:25:44.213 [DEBUG] [org.apache.http.headers] << Date: Fri, 10 May 2013 01:25:44 GMT
11:25:44.214 [DEBUG] [org.apache.http.headers] << Server: Apache
11:25:44.214 [DEBUG] [org.apache.http.headers] << Content-Length: 270
11:25:44.214 [DEBUG] [org.apache.http.headers] << Content-Type: text/html; charset=iso-8859-1
11:25:44.214 [DEBUG] [org.apache.http.headers] << Proxy-Connection: Keep-alive
11:25:44.214 [DEBUG] [org.apache.http.impl.client.SystemDefaultHttpClient] Connection can be kept alive indefinitely
this a sample of my builde script:
repositories {
mavenCentral()
maven { url "http://owasp-java-html-sanitizer.googlecode.com/svn/maven"}
maven { url "http://www.terracotta.org/download/reflector/releases"}
}
dependencies{
....
compile "owasp-java-html-sanitizer:owasp-java-html-sanitizer:r99"
compile "net.sf.ehcache:ehcache-ee:2.7.0"
compile "org.terracotta.bigmemory:bigmemory:4.0.0"
...
}
If I change the order of the repo, the problem is the same but is stuck on different dependency…
So far my build is running for 30 minutes…
PS: I have tried with gradle 1.4 and now with gradle 1.6. Same problem… PS2 : If I hit http://owasp-java-html-sanitizer.googlecode.com/svn/maven/net/sf/ehcache/ehcache-ee-root/2.7.0/ehcache-ee-root-2.7.0.pom manually, It returns 404 straight as expected. So there is absolutely no reason for gradle to be stuck, The servers are very reactive
One reason is that Maven supports parallel dependency resolution, which is currently a feature request GRADLE-2582).
If you have many dependencies then it becomes worthwhile having an on-site artifact repository. That provides a caching proxy and minimizes disruptions if the remote repository disappears. I have gradle fetch only from local Artifactory, which has all of the remote repositories proxied. The builds went from 15 minutes to 30 seconds. Slow dependency downloading became a non-issue at that point.
1 Like
I can’t reproduce this. If I use the build snippet above, a clean resolution (i.e. --refresh-dependencies) takes 6 secs.
Are you in a proxy environment? You likely have some network impediment between you and the host.
I am not in a proxy environment. The thing is that my network connection doesn’t have any problem and the problem was always on 2 completely different servers (the ones declared in the snippet) while all the others were OK. I have tested manually those servers and everything was OK.
I am going to try from home and let you know. Maybe the problem is due to the rest of the build script… If I can reproduce, I will try to provide you a full sample project where the problem occurs and give you at least a stack trace of the gradle process when it is stuck.
I just tried from home. everything run without any problem.
I have no idea why it wasn’t working at work. I will retry on Monday to see if the problem is still there.
Sorry for the false alert :S
The problem has happened again at work. I have done some debugging work and I have found the problem. It comes from the server. You can simulate the request done by gradle :
curl -v -XHEAD http://owasp-java-html-sanitizer.googlecode.com/svn/maven/org/terracotta/bigmemory/bigmemory/4.0.0/bigmemory-4.0.0.pom
You will see that curl is also stuck. That server response :
HTTP/1.1 404 Not Found
Date: Tue, 14 May 2013 03:11:40 GMT
Server: Apache
Content-Type: text/html; charset=iso-8859-1
Transfer-Encoding: chunked
Basically, only when the request method is HEAD, the server return “Transfer-Encoding: chunked” Which is supposed to mean that the body will be sent in chunk. Except that the server never send the body and so gradle (as well as curl) is stuck, waiting for the body.
I have tested on both Mac and linux and the problem is the same. I don’t have a windows platform to test.
One workaround would be do a GET request instead of HEAD, but I guess it has other consequences…
I have only 3 repositories (mvnCentral excluded) and 2 of them do the same! That’s a real pain!
The outputs of commands used : https://gist.github.com/thibaultdelor/5573497
The stack trace of gradle when stucked : https://gist.github.com/thibaultdelor/5573506
I have no idea why this bug doesn’t happen on all platforms…
Ah, I didn’t pick it up that it was a Google Code URL.
Google Code is broken for HEAD requests: https://code.google.com/p/support/issues/detail?id=660
Their bug is intermittent, which is why you are only seeing it sometimes.
There’s nothing we can really do here as Google Code is clearly in the wrong. I suggest contacting the people producing the library and encourage them to move to a proper binary hosting solution such as http://bintray.com/
1 Like