Build performance degrades extremely rapidly when using excludes to manage dependencies


(Cameron Fieber) #1

A build that has 10 all*.exclude and 10 compile.exclude is taking 30 minutes to resolve the dependencies against a fully populated local cache. The behaviour is present in 1.7, 1.8-rc-1, and 1.9 nightly

As a workaround, I mimic the behaviour of exclude by using a resolutionStrategy that, if the DependencyResolveDetails matches one of my exclusions, I just set useTarget to some known good dependency (in this case remap all excludes to slf4j-api). With 1.9 nightly the same build resolves in under 30 seconds against a populated local cache.

Here is my workaround in case anyone runs into the same problem:

configurations {

def fakeExclude = { List excludes, DependencyResolveDetails details ->

def replacement = ‘org.slf4j:slf4j-api:1.7.2’

def translate = { k ->

if (k == ‘module’) {

‘name’

} else {

k

}

}

if (excludes.find {

boolean match = true

it.each { k,v ->

match &= details.requested.properties[translate(k)] == v

}

match

}) {

details.useTarget(replacement)

}

}

all {

resolutionStrategy {

def excludes = [

[ module: “jersey-bundle” ]

]

eachDependency { DependencyResolveDetails details ->

fakeExclude(excludes, details)

}

}

}

compile {

resolutionStrategy {

def excludes = [

[ module: “ant” ]

]

eachDependency { DependencyResolveDetails details ->

fakeExclude(excludes, details)

}

}

}

}


(Cameron Fieber) #2

It seems like there is more at play than just the number of excludes. The projects that were exhibiting the insane resolve times also rely pretty heavily on dynamic versions for their dependencies (probably 10 or so dependencies with dynamic versions [e.g. latest.])


(Peter Niederwieser) #3

Use of dynamic revisions can slow down resolution considerably. Also make sure that your repository manager isn’t a bottleneck.