Error while downloading dependencies from Amazon S3

Old Forum Archive Bugs
,
1

Gradle Version: 2.14.1
Operating System: Windows 7 x64, Ubuntu 14.04

We have some big projects with a lot of modules, which we publish and download from Amazon S3.
After we publish new versions of modules our builds often fail with this error:
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool

Stacktrace:

[INFO] [com.amazonaws.http.AmazonHttpClient] Unable to execute HTTP request: Timeout waiting for connection from pool
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
	at org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:226)
	at org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:195)
	at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)
	at com.amazonaws.http.conn.$Proxy78.getConnection(Unknown Source)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:423)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:685)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:460)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:295)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3736)
	at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1163)
	at org.gradle.internal.resource.transport.aws.s3.S3Client.doGetS3Object(S3Client.java:182)
	at org.gradle.internal.resource.transport.aws.s3.S3Client.getMetaData(S3Client.java:114)
	at org.gradle.internal.resource.transport.aws.s3.S3ResourceConnector.getMetaData(S3ResourceConnector.java:59)
	at org.gradle.internal.resource.transport.DefaultExternalResourceRepository.getResourceMetaData(DefaultExternalResourceRepository.java:68)
	at org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver.staticResourceExists(DefaultExternalResourceArtifactResolver.java:69)
	at org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver.artifactExists(DefaultExternalResourceArtifactResolver.java:60)
	at org.gradle.api.internal.artifacts.repositories.resolver.MavenResolver$MavenRemoteRepositoryAccess.resolveConfigurationArtifacts(MavenResolver.java:250)
	at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver$AbstractRepositoryAccess.resolveModuleArtifacts(ExternalResourceResolver.java:363)
	at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver$RemoteRepositoryAccess.resolveModuleArtifacts(ExternalResourceResolver.java:420)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CacheLockReleasingModuleComponentsRepository$LockReleasingRepositoryAccess$4.run(CacheLockReleasingModuleComponentsRepository.java:87)
	at org.gradle.internal.Factories$1.create(Factories.java:22)
	at org.gradle.cache.internal.DefaultCacheAccess.longRunningOperation(DefaultCacheAccess.java:242)
	at org.gradle.cache.internal.DefaultCacheAccess.longRunningOperation(DefaultCacheAccess.java:313)
	at org.gradle.cache.internal.DefaultPersistentDirectoryStore.longRunningOperation(DefaultPersistentDirectoryStore.java:114)
	at org.gradle.cache.internal.DefaultCacheFactory$ReferenceTrackingCache.longRunningOperation(DefaultCacheFactory.java:179)
	at org.gradle.api.internal.artifacts.ivyservice.DefaultCacheLockingManager.longRunningOperation(DefaultCacheLockingManager.java:56)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CacheLockReleasingModuleComponentsRepository$LockReleasingRepositoryAccess.resolveModuleArtifacts(CacheLockReleasingModuleComponentsRepository.java:85)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository$ResolveAndCacheRepositoryAccess.resolveModuleArtifacts(CachingModuleComponentRepository.java:327)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.BaseModuleComponentRepositoryAccess.resolveModuleArtifacts(BaseModuleComponentRepositoryAccess.java:50)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess.resolveModuleArtifacts(ErrorHandlingModuleComponentRepository.java:107)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainArtifactResolver.resolveModuleArtifacts(RepositoryChainArtifactResolver.java:54)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.ComponentResolversChain$ArtifactResolverChain.resolveModuleArtifacts(ComponentResolversChain.java:108)
	at org.gradle.api.internal.artifacts.ivyservice.ContextualArtifactResolver$2.execute(ContextualArtifactResolver.java:53)
	at org.gradle.api.internal.artifacts.ivyservice.ContextualArtifactResolver$2.execute(ContextualArtifactResolver.java:51)
	at org.gradle.internal.Transformers$4.transform(Transformers.java:169)
	at org.gradle.api.internal.artifacts.ivyservice.DefaultIvyContextManager.withIvy(DefaultIvyContextManager.java:48)
	at org.gradle.api.internal.artifacts.ivyservice.DefaultIvyContextManager.withIvy(DefaultIvyContextManager.java:39)
	at org.gradle.api.internal.artifacts.ivyservice.ContextualArtifactResolver$4.run(ContextualArtifactResolver.java:69)
	at org.gradle.internal.Factories$1.create(Factories.java:22)
	at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:192)
	at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:175)
	at org.gradle.cache.internal.DefaultPersistentDirectoryStore.useCache(DefaultPersistentDirectoryStore.java:106)
	at org.gradle.cache.internal.DefaultCacheFactory$ReferenceTrackingCache.useCache(DefaultCacheFactory.java:187)
	at org.gradle.api.internal.artifacts.ivyservice.DefaultCacheLockingManager.useCache(DefaultCacheLockingManager.java:64)
	at org.gradle.api.internal.artifacts.ivyservice.ContextualArtifactResolver.executeInContext(ContextualArtifactResolver.java:67)
	at org.gradle.api.internal.artifacts.ivyservice.ContextualArtifactResolver.resolveModuleArtifacts(ContextualArtifactResolver.java:51)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingArtifactResolver.resolveModuleArtifacts(ErrorHandlingArtifactResolver.java:45)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.artifact.ResolvedArtifactsGraphVisitor.doResolve(ResolvedArtifactsGraphVisitor.java:108)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.artifact.ResolvedArtifactsGraphVisitor.getArtifacts(ResolvedArtifactsGraphVisitor.java:92)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.artifact.ResolvedArtifactsGraphVisitor.visitEdge(ResolvedArtifactsGraphVisitor.java:68)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.CompositeDependencyGraphVisitor.visitEdge(CompositeDependencyGraphVisitor.java:43)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.DependencyGraphBuilder.assembleResult(DependencyGraphBuilder.java:177)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.DependencyGraphBuilder.resolve(DependencyGraphBuilder.java:81)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.DefaultArtifactDependencyResolver$1.execute(DefaultArtifactDependencyResolver.java:90)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.DefaultArtifactDependencyResolver$1.execute(DefaultArtifactDependencyResolver.java:80)
	at org.gradle.internal.Transformers$4.transform(Transformers.java:169)
	at org.gradle.api.internal.artifacts.ivyservice.DefaultIvyContextManager.withIvy(DefaultIvyContextManager.java:61)
	at org.gradle.api.internal.artifacts.ivyservice.DefaultIvyContextManager.withIvy(DefaultIvyContextManager.java:39)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.DefaultArtifactDependencyResolver.resolve(DefaultArtifactDependencyResolver.java:80)
	at org.gradle.api.internal.artifacts.ivyservice.CacheLockingArtifactDependencyResolver$1.run(CacheLockingArtifactDependencyResolver.java:41)
	at org.gradle.internal.Factories$1.create(Factories.java:22)
	at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:192)
	at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:175)
	at org.gradle.cache.internal.DefaultPersistentDirectoryStore.useCache(DefaultPersistentDirectoryStore.java:106)
	at org.gradle.cache.internal.DefaultCacheFactory$ReferenceTrackingCache.useCache(DefaultCacheFactory.java:187)
	at org.gradle.api.internal.artifacts.ivyservice.DefaultCacheLockingManager.useCache(DefaultCacheLockingManager.java:64)
	at org.gradle.api.internal.artifacts.ivyservice.CacheLockingArtifactDependencyResolver.resolve(CacheLockingArtifactDependencyResolver.java:39)
	at org.gradle.api.internal.artifacts.ivyservice.DefaultConfigurationResolver.resolve(DefaultConfigurationResolver.java:91)
	at org.gradle.api.internal.artifacts.ivyservice.SelfResolvingDependencyConfigurationResolver.resolve(SelfResolvingDependencyConfigurationResolver.java:40)
	at org.gradle.api.internal.artifacts.ivyservice.ShortCircuitEmptyConfigurationResolver.resolve(ShortCircuitEmptyConfigurationResolver.java:52)
	at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingConfigurationResolver.resolve(ErrorHandlingConfigurationResolver.java:43)
	at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveGraphIfRequired(DefaultConfiguration.java:372)
	at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveNow(DefaultConfiguration.java:347)
	at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.getResolvedConfiguration(DefaultConfiguration.java:340)
	at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration_Decorated.getResolvedConfiguration(Unknown Source)
	at com.android.build.gradle.internal.DependencyManager.collectArtifacts(DependencyManager.java:747)
	at com.android.build.gradle.internal.DependencyManager.resolveDependencyForConfig(DependencyManager.java:255)
	at com.android.build.gradle.internal.DependencyManager.resolveDependencies(DependencyManager.java:139)
	at com.android.build.gradle.internal.TaskManager.resolveDependencies(TaskManager.java:356)
	at com.android.build.gradle.internal.VariantManager$9.call(VariantManager.java:597)
	at com.android.build.gradle.internal.VariantManager$9.call(VariantManager.java:594)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:55)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:47)
	at com.android.build.gradle.internal.profile.SpanRecorders.record(SpanRecorders.java:51)
	at com.android.build.gradle.internal.VariantManager.createVariantData(VariantManager.java:593)
	at com.android.build.gradle.internal.VariantManager.createVariantDataForProductFlavors(VariantManager.java:750)
	at com.android.build.gradle.internal.VariantManager.populateVariantDataList(VariantManager.java:459)
	at com.android.build.gradle.internal.VariantManager$1.call(VariantManager.java:263)
	at com.android.build.gradle.internal.VariantManager$1.call(VariantManager.java:260)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:55)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:47)
	at com.android.build.gradle.internal.VariantManager.createAndroidTasks(VariantManager.java:259)
	at com.android.build.gradle.BasePlugin$12.call(BasePlugin.java:657)
	at com.android.build.gradle.BasePlugin$12.call(BasePlugin.java:654)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:55)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:47)
	at com.android.build.gradle.BasePlugin.createAndroidTasks(BasePlugin.java:653)
	at com.android.build.gradle.BasePlugin$10$1.call(BasePlugin.java:572)
	at com.android.build.gradle.BasePlugin$10$1.call(BasePlugin.java:569)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:55)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:47)
	at com.android.build.gradle.BasePlugin$10.execute(BasePlugin.java:568)
	at com.android.build.gradle.BasePlugin$10.execute(BasePlugin.java:565)
	at org.gradle.internal.event.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:93)
	at org.gradle.internal.event.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:82)
	at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:44)
	at org.gradle.internal.event.BroadcastDispatch.dispatch(BroadcastDispatch.java:79)
	at org.gradle.internal.event.BroadcastDispatch.dispatch(BroadcastDispatch.java:30)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
	at com.sun.proxy.$Proxy11.afterEvaluate(Unknown Source)
	at org.gradle.configuration.project.LifecycleProjectEvaluator.notifyAfterEvaluate(LifecycleProjectEvaluator.java:67)
	at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:61)
	at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:540)
	at org.gradle.api.internal.project.AbstractProject.evaluationDependsOn(AbstractProject.java:612)
	at org.gradle.api.internal.project.AbstractProject.evaluationDependsOn(AbstractProject.java:604)
	at com.android.build.gradle.internal.DependencyManager.ensureConfigured(DependencyManager.java:724)
	at com.android.build.gradle.internal.DependencyManager.resolveDependencyForConfig(DependencyManager.java:240)
	at com.android.build.gradle.internal.DependencyManager.resolveDependencies(DependencyManager.java:139)
	at com.android.build.gradle.internal.TaskManager.resolveDependencies(TaskManager.java:356)
	at com.android.build.gradle.internal.VariantManager$9.call(VariantManager.java:597)
	at com.android.build.gradle.internal.VariantManager$9.call(VariantManager.java:594)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:55)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:47)
	at com.android.build.gradle.internal.profile.SpanRecorders.record(SpanRecorders.java:51)
	at com.android.build.gradle.internal.VariantManager.createVariantData(VariantManager.java:593)
	at com.android.build.gradle.internal.VariantManager.createVariantDataForProductFlavors(VariantManager.java:750)
	at com.android.build.gradle.internal.VariantManager.populateVariantDataList(VariantManager.java:459)
	at com.android.build.gradle.internal.VariantManager$1.call(VariantManager.java:263)
	at com.android.build.gradle.internal.VariantManager$1.call(VariantManager.java:260)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:55)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:47)
	at com.android.build.gradle.internal.VariantManager.createAndroidTasks(VariantManager.java:259)
	at com.android.build.gradle.BasePlugin$12.call(BasePlugin.java:657)
	at com.android.build.gradle.BasePlugin$12.call(BasePlugin.java:654)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:55)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:47)
	at com.android.build.gradle.BasePlugin.createAndroidTasks(BasePlugin.java:653)
	at com.android.build.gradle.BasePlugin$10$1.call(BasePlugin.java:572)
	at com.android.build.gradle.BasePlugin$10$1.call(BasePlugin.java:569)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:55)
	at com.android.builder.profile.ThreadRecorder$1.record(ThreadRecorder.java:47)
	at com.android.build.gradle.BasePlugin$10.execute(BasePlugin.java:568)
	at com.android.build.gradle.BasePlugin$10.execute(BasePlugin.java:565)
	at org.gradle.internal.event.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:93)
	at org.gradle.internal.event.BroadcastDispatch$ActionInvocationHandler.dispatch(BroadcastDispatch.java:82)
	at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:44)
	at org.gradle.internal.event.BroadcastDispatch.dispatch(BroadcastDispatch.java:79)
	at org.gradle.internal.event.BroadcastDispatch.dispatch(BroadcastDispatch.java:30)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
	at com.sun.proxy.$Proxy11.afterEvaluate(Unknown Source)
	at org.gradle.configuration.project.LifecycleProjectEvaluator.notifyAfterEvaluate(LifecycleProjectEvaluator.java:67)
	at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:61)
	at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:540)
	at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:93)
	at org.gradle.execution.TaskPathProjectEvaluator.configureHierarchy(TaskPathProjectEvaluator.java:47)
	at org.gradle.configuration.DefaultBuildConfigurer.configure(DefaultBuildConfigurer.java:35)
	at org.gradle.initialization.DefaultGradleLauncher$2.run(DefaultGradleLauncher.java:124)
	at org.gradle.internal.Factories$1.create(Factories.java:22)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:53)
	at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:121)
	at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:32)
	at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:98)
	at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:92)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:63)
	at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:92)
	at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:83)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:99)
	at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
	at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:48)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:30)
	at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:81)
	at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:46)
	at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:52)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:37)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:26)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:34)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:74)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:72)
	at org.gradle.util.Swapper.swap(Swapper.java:38)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:72)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.health.DaemonHealthTracker.execute(DaemonHealthTracker.java:47)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:72)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.health.HintGCAfterBuild.execute(HintGCAfterBuild.java:41)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
	at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:237)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

There are also some messages in build log (–debug) like these:
[DEBUG] [org.apache.http.impl.conn.PoolingClientConnectionManager] Connection request: [route: {s}->https://***.s3.amazonaws.com:443][total kept alive: 0; route allocated: 49 of 50; total allocated: 49 of 50]
[DEBUG] [org.apache.http.impl.conn.PoolingClientConnectionManager] Connection leased: [id: 83][route: {s}->https://***.s3.amazonaws.com:443][total kept alive: 0; route allocated: 50 of 50; total allocated: 50 of 50]

So there is definitely S3Object connection leak somewhere. Please review code and verify that all instances of S3Object are closed.

2

Thanks for the report. I’ll take a closer look on this and let you know about the findings.
You can follow the progress also by tracking Error while downloading dependencies from Amazon S3
cheers,
René

3

Hello Nikolai,

Can you try out 3.0-milestone-2 and see if this fixes the problem you described?

If not, can you provide some more details about your build? I tried to reproduce what you explained by publishing and resolving multiple 1000 artifacts from/via s3 storage but can’t this this behaviour.

I assume you’re using ivy? How do you declare your dependencies? With fixed versions or dynamic versions (1.+, latest.integration) or possibly as changing modules (e.g. 1.x-SNAPSHOTS)?

4

Hi Rene,

Well, I’m pretty sure we are using Maven, not Ivy.

Here is a sample of our publishing config which we include in all modules:

publishing {
    publications {
        maven(MavenPublication) {
            groupId = project.group
            artifactId = project.name
            version = project.version
    }

    repositories {
        maven {
            url "s3://ourCompany"
            credentials(AwsCredentials) {
                accessKey "***"
                secretKey "***"
            }
        }
    }
}

And we consume dependencies like this:

allprojects {
    repositories {
        jcenter() // for 3-d party libs

        maven {
            url "s3://ourCompany"
            credentials(AwsCredentials) {
                accessKey "***"
                secretKey "***"
            }
        }
    }
}
dependencies {
    compile 'company:product:2.4.0'
    ...
}

We always use fixed versions of dependencies.

Some strange things I see in the build log (–debug):
Dependencies start downloading normally with code 200, and connections in pool are released after every request:
[DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Receiving response: HTTP/1.1 200 OK
[DEBUG] [org.apache.http.impl.conn.PoolingClientConnectionManager] Connection released: [id: 0][route: {s}->https://***.s3.amazonaws.com][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]

At some point log gets flooded with
[DEBUG] [org.apache.http.impl.conn.DefaultClientConnection] Receiving response: HTTP/1.1 206 Partial Content
And connection pools grows rapidly:
[DEBUG] [org.apache.http.impl.conn.PoolingClientConnectionManager] Connection request: [route: {s}->https://***.s3.amazonaws.com][total kept alive: 0; route allocated: 8 of 50; total allocated: 8 of 50]
Shortly it becomes 50 of 50 and then build breaks with pool timeout.
So maybe there is a problem with releasing connections after 206?

I tried to assemble project with 3.0 milestone 2, but build fails due to some Android plugin stuff (and yes, I inserted System.properties[‘com.android.build.gradle.overrideVersionCheck’] = ‘true’ in buildscript).

5

Thanks for further details. I’ll look into this again, trying to reproduce the 206 behaviour somehow you’re seeing.

6

Still trying reproducing this. When you say “at one point” do you mean in the same build you see the “normal” behaviour of 200 requests that are correctly released afterwards or do you mean in a later build. For receiving metadata we set the download range to 0 to mimic head requests. The connection handling is completely done within the java-aws-sdk library, so at the moment I think the culprit is somewhere in there. still digging…

7

Yes, it’s all in the same build.

8

The problem is fixed for me in 3.1 RC1. Thank you!