Dependency resolving became very slow


(Zsolt Kúti) #1

Hello,

My project’s compilation process suddenly got very slow today. In debug mode I see gradle (1.0-rc-3) querying a repo which is presently working strange. Below is a snippet from the log. If I understand well what is going on gradle tries nexus.bedatadriven.com for dependencies, which gives erronous answer (after a long time), then gets dependencies from elsewhere. This keeps repeated for all the dependencies putting effectively into halt the process. I tried to increase some cache timeouts with no effect: resolutionStrategy.cacheDynamicVersionsFor 60, ‘days’ resolutionStrategy.cacheChangingModulesFor 60, ‘days’

Is there a way to avoid the above slow-down?

Thanks! Zsolt

08:49:27.892 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Visiting dependency hu.prolan.kvf#web-ui;0.1.0(compile) -> javax#mail;1.4.4([default]) 08:49:27.922 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Selecting new module version javax#mail;1.4.4 08:49:27.923 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.UserResolverChain] Attempting to resolve module ‘javax#mail;1.4.4’ using repositories ‘[prolanRepo, MavenRepo, maven, maven2]’ 08:49:27.927 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on artifact cache (/usr/home/kuti/.gradle/caches/artifacts-13). 08:49:27.928 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired. 08:49:27.928 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache dynamic-revisions.bin (/usr/home/kuti/.gradle/caches/artifacts-13/dynamic-revisions.bin) 08:49:27.954 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache module-metadata.bin (/usr/home/kuti/.gradle/caches/artifacts-13/module-metadata.bin) 08:49:27.955 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.IvyLoggingAdaper] post 1.3 ivy file: using exact as default matcher 08:49:27.957 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleVersionRepository] Using cached module metadata for module ‘javax#mail;1.4.4’ in ‘prolanRepo’ 08:49:27.958 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleVersionRepository] Detected non-existence of module ‘javax#mail;1.4.4’ in resolver cache ‘MavenRepo’ 08:49:27.958 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleVersionRepository] Detected non-existence of module ‘javax#mail;1.4.4’ in resolver cache ‘maven’ 08:49:27.958 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleVersionRepository] Cached meta-data for missing module is expired: will perform fresh resolve of ‘javax#mail;1.4.4’ in ‘maven2’ 08:49:27.959 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache module-metadata.bin (/usr/home/kuti/.gradle/caches/artifacts-13/module-metadata.bin) 08:49:27.985 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache dynamic-revisions.bin (/usr/home/kuti/.gradle/caches/artifacts-13/dynamic-revisions.bin) 08:49:27.989 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on artifact cache (/usr/home/kuti/.gradle/caches/artifacts-13). 08:49:27.992 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.IvyLoggingAdaper]

tried http://nexus.bedatadriven.com/content/groups/public/javax/mail/1.4.4/mail-1.4.4.pom 08:49:27.992 [DEBUG] [org.gradle.api.internal.artifacts.repositories.ExternalResourceResolver] Loading http://nexus.bedatadriven.com/content/groups/public/javax/mail/1.4.4/mail-1.4.4.pom 08:49:27.993 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on artifact cache (/usr/home/kuti/.gradle/caches/artifacts-13). 08:49:27.993 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired. 08:49:28.016 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache artifact-at-url.bin (/usr/home/kuti/.gradle/caches/artifacts-13/artifact-at-url.bin) 08:49:28.021 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache artifact-at-url.bin (/usr/home/kuti/.gradle/caches/artifacts-13/artifact-at-url.bin) 08:49:28.021 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on artifact cache (/usr/home/kuti/.gradle/caches/artifacts-13). 08:49:28.022 [DEBUG] [org.gradle.api.internal.externalresource.transfer.DefaultCacheAwareExternalResourceAccessor] Constructing external resource: http://nexus.bedatadriven.com/content/groups/public/javax/mail/1.4.4/mail-1.4.4.pom 08:49:28.025 [DEBUG] [org.gradle.api.internal.externalresource.transport.http.HttpResourceAccessor] Constructing external resource: http://nexus.bedatadriven.com/content/groups/public/javax/mail/1.4.4/mail-1.4.4.pom 08:49:28.047 [DEBUG] [org.gradle.api.internal.externalresource.transport.http.HttpClientHelper] Performing HTTP GET: http://nexus.bedatadriven.com/content/groups/public/javax/mail/1.4.4/mail-1.4.4.pom 08:49:28.051 [DEBUG] [org.apache.http.impl.conn.SingleClientConnManager] Get connection for route HttpRoute[{}->http://proxy:3128->http://nexus.bedatadriven.com] 08:49:28.072 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnectionOperator] Connecting to proxy:3128 08:49:28.073 [DEBUG] [org.apache.http.client.protocol.RequestAddCookies] CookieSpec selected: best-match 08:49:28.073 [DEBUG] [org.apache.http.client.protocol.RequestAuthCache] Auth cache not set in the context 08:49:28.078 [DEBUG] [org.apache.http.impl.client.ContentEncodingHttpClient] Attempt 1 to execute request 08:49:28.081 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Sending request: GET http://nexus.bedatadriven.com/content/groups/public/javax/mail/1.4.4/mail-1.4.4.pom HTTP/1.1 08:49:28.082 [DEBUG] [org.apache.http.headers] >> GET http://nexus.bedatadriven.com/content/groups/public/javax/mail/1.4.4/mail-1.4.4.pom HTTP/1.1 08:49:28.083 [DEBUG] [org.apache.http.headers] >> User-Agent: Gradle/1.0-rc-3 08:49:28.083 [DEBUG] [org.apache.http.headers] >> Host: nexus.bedatadriven.com 08:49:28.084 [DEBUG] [org.apache.http.headers] >> Proxy-Connection: Keep-Alive 08:49:28.084 [DEBUG] [org.apache.http.headers] >> Accept-Encoding: gzip,deflate 08:53:58.833 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Receiving response: HTTP/1.0 502 Bad Gateway 08:53:58.834 [DEBUG] [org.apache.http.headers] << HTTP/1.0 502 Bad Gateway 08:53:58.834 [DEBUG] [org.apache.http.headers] << Server: squid/3.0.STABLE16 08:53:58.834 [DEBUG] [org.apache.http.headers] << Mime-Version: 1.0 08:53:58.834 [DEBUG] [org.apache.http.headers] << Date: Thu, 31 May 2012 06:53:58 GMT 08:53:58.834 [DEBUG] [org.apache.http.headers] << Content-Type: text/html 08:53:58.844 [DEBUG] [org.apache.http.headers] << Content-Length: 1278 08:53:58.845 [DEBUG] [org.apache.http.headers] << X-Squid-Error: ERR_INVALID_RESP 0 08:53:58.848 [DEBUG] [org.apache.http.headers] << X-Cache: MISS from viruswall 08:53:58.848 [DEBUG] [org.apache.http.headers] << X-Cache-Lookup: MISS from viruswall:3128 08:53:58.848 [DEBUG] [org.apache.http.headers] << Via: 1.0 viruswall (squid/3.0.STABLE16) 08:53:58.848 [DEBUG] [org.apache.http.headers] << Proxy-Connection: close 08:53:58.860 [DEBUG] [org.apache.http.impl.conn.SingleClientConnManager] Releasing connection org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter@1496198 08:53:58.861 [DEBUG] [org.apache.http.impl.conn.SingleClientConnManager] Released connection open but not reusable. 08:53:58.863 [DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Connection shut down 08:53:58.864 [INFO] [org.gradle.api.internal.externalresource.transport.http.HttpClientHelper] Failed to get resource: GET. [HTTP HTTP/1.0 502 Bad Gateway: http://nexus.bedatadriven.com/content/groups/public/javax/mail/1.4.4/mail-1.4.4.pom] 08:53:58.864 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.UserResolverChain] Using module ‘javax#mail;1.4.4’ from repository ‘prolanRepo’


(Zsolt Kúti) #2

FYI.

As a workaround I commented out this repo and found another one for the given dependency.

Still interested what can do gradle in such case.


(Luke Daley) #3

One thing we could do is download dependencies in parallel batches.