Inconsistent Gradle Dependency Resolution

varidvaya · February 23, 2024, 6:16pm

I’m receiving error “Did not resolve ‘ch.randelshofer:fastdoubleparser:0.8.0’ which is part of the dependency lock state” during Gradle project build in Jenkins server. I am using Gradle dependency locking and write the lockfile in my local machine. I’ve tried to match the environment of my local machine to the Jenkins server i.e., matching the Java and Gradle version. However there are some specification that I can’t match e.g., OS (Windows/Ubuntu WSL to RedHat).

The weird thing is, my local machine and Jenkins resolve some of the dependencies differently. In this case 3 library from about 200 dependencies, the other just resolved consistently. These dependencies: ch.randelshofer:fastdoubleparser, org.junit:junit-bom and org.apiguardian:apiguardian-api weren’t resolved by Jenkins. I’ve tried in different local machine, all of them resolve just like mine.

I’ve tried run it on different Java (17 and 21) and Gradle (8.1.1, 8.2 and 8.2.1) version, different OS (Windows and Ubuntu WSL), different laptop, they all are consistent. I’ve also tried clearing cache. Describing the Jenkins server, it’s using Java 17, Gradle 8.1.1 and OS RedHat.

So, why is the Jenkins resolve it differently?

I’ve tried to list the dependency tree as the result can be seen below. To help you focus on the branch that is different, I’ve put marks FOCUS_ON_ME!!! right on their side.

Dependecy tree from my local machine

...
compileClasspath - Compile classpath for source set 'main'.
+--- org.springframework.boot:spring-boot-starter-web:3.1.0
|    +--- org.springframework.boot:spring-boot-starter:3.1.0 (*)
|    +--- org.springframework.boot:spring-boot-starter-json:3.1.0
|    |    +--- org.springframework.boot:spring-boot-starter:3.1.0 (*)
|    |    +--- org.springframework:spring-web:6.0.9
|    |    |    +--- org.springframework:spring-beans:6.0.9 (*)
|    |    |    +--- org.springframework:spring-core:6.0.9 (*)
|    |    |    \--- io.micrometer:micrometer-observation:1.10.7 -> 1.11.0
|    |    |         \--- io.micrometer:micrometer-commons:1.11.0
|    |    +--- com.fasterxml.jackson.core:jackson-databind:2.15.0
|    |    |    +--- com.fasterxml.jackson.core:jackson-annotations:2.15.0
|    |    |    |    \--- com.fasterxml.jackson:jackson-bom:2.15.0
|    |    |    |         +--- com.fasterxml.jackson.core:jackson-annotations:2.15.0 (c)
|    |    |    |         +--- com.fasterxml.jackson.core:jackson-core:2.15.0 (c)
|    |    |    |         +--- com.fasterxml.jackson.core:jackson-databind:2.15.0 (c)
|    |    |    |         +--- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.15.0 (c)
|    |    |    |         +--- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.15.0 (c)
|    |    |    |         +--- com.fasterxml.jackson.module:jackson-module-parameter-names:2.15.0 (c)
|    |    |    |         \--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.15.0 (c)
|    |    |    +--- com.fasterxml.jackson.core:jackson-core:2.15.0 FOCUS_ON_ME!!!
|    |    |    |    +--- ch.randelshofer:fastdoubleparser:0.8.0
|    |    |    |    \--- com.fasterxml.jackson:jackson-bom:2.15.0 (*)
|    |    |    \--- com.fasterxml.jackson:jackson-bom:2.15.0 (*)
|    |    +--- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.15.0

testRuntimeClasspath - Runtime classpath of source set 'test'.
+--- org.springframework.boot:spring-boot-starter-test:3.1.0
|    +--- org.junit.jupiter:junit-jupiter:5.9.3
|    |    +--- org.junit.jupiter:junit-jupiter-api:5.9.3 FOCUS_ON_ME!!!
|    |    |    +--- org.opentest4j:opentest4j:1.2.0
|    |    |    +--- org.junit.platform:junit-platform-commons:1.9.3 FOCUS_ON_ME!!!
|    |    |    |    \--- org.apiguardian:apiguardian-api:1.1.2
|    |    |    \--- org.apiguardian:apiguardian-api:1.1.2
|    |    +--- org.junit.jupiter:junit-jupiter-params:5.9.3 FOCUS_ON_ME!!!
|    |    |    +--- org.junit.jupiter:junit-jupiter-api:5.9.3 (*)
|    |    |    \--- org.apiguardian:apiguardian-api:1.1.2
|    |    \--- org.junit.jupiter:junit-jupiter-engine:5.9.3 FOCUS_ON_ME!!!
|    |         +--- org.junit.platform:junit-platform-engine:1.9.3 FOCUS_ON_ME!!!
|    |         |    +--- org.opentest4j:opentest4j:1.2.0
|    |         |    +--- org.junit.platform:junit-platform-commons:1.9.3 (*)
|    |         |    \--- org.apiguardian:apiguardian-api:1.1.2
|    |         +--- org.junit.jupiter:junit-jupiter-api:5.9.3 (*)
|    |         \--- org.apiguardian:apiguardian-api:1.1.2
|    +--- org.mockito:mockito-core:5.3.1
...

Dependency tree from Jenkins server

...
compileClasspath - Compile classpath for source set 'main'.
+--- org.springframework.boot:spring-boot-starter-web:3.1.0
|    +--- org.springframework.boot:spring-boot-starter:3.1.0 (*)
|    +--- org.springframework.boot:spring-boot-starter-json:3.1.0
|    |    +--- org.springframework.boot:spring-boot-starter:3.1.0 (*)
|    |    +--- org.springframework:spring-web:6.0.9
|    |    |    +--- org.springframework:spring-beans:6.0.9 (*)
|    |    |    +--- org.springframework:spring-core:6.0.9 (*)
|    |    |    \--- io.micrometer:micrometer-observation:1.10.7 -> 1.11.0
|    |    |         \--- io.micrometer:micrometer-commons:1.11.0
|    |    +--- com.fasterxml.jackson.core:jackson-databind:2.15.0
|    |    |    +--- com.fasterxml.jackson.core:jackson-annotations:2.15.0
|    |    |    \--- com.fasterxml.jackson.core:jackson-core:2.15.0 FOCUS_ON_ME!!!
|    |    +--- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.15.0

testRuntimeClasspath - Runtime classpath of source set 'test'.
+--- org.springframework.boot:spring-boot-starter-test:3.1.0
|    +--- org.junit.jupiter:junit-jupiter:5.9.3
|    |    +--- org.junit:junit-bom:5.9.3
|    |    |    +--- org.junit.jupiter:junit-jupiter:5.9.3 (c)
|    |    |    +--- org.junit.jupiter:junit-jupiter-api:5.9.3 (c)
|    |    |    +--- org.junit.jupiter:junit-jupiter-engine:5.9.3 (c)
|    |    |    +--- org.junit.jupiter:junit-jupiter-params:5.9.3 (c)
|    |    |    +--- org.junit.platform:junit-platform-commons:1.9.3 (c)
|    |    |    \--- org.junit.platform:junit-platform-engine:1.9.3 (c)
|    |    +--- org.junit.jupiter:junit-jupiter-api:5.9.3 FOCUS_ON_ME!!!
|    |    |    +--- org.junit:junit-bom:5.9.3 (*)
|    |    |    +--- org.opentest4j:opentest4j:1.2.0
|    |    |    \--- org.junit.platform:junit-platform-commons:1.9.3 FOCUS_ON_ME!!!
|    |    |         \--- org.junit:junit-bom:5.9.3 (*)
|    |    +--- org.junit.jupiter:junit-jupiter-params:5.9.3 FOCUS_ON_ME!!!
|    |    |    +--- org.junit:junit-bom:5.9.3 (*)
|    |    |    \--- org.junit.jupiter:junit-jupiter-api:5.9.3 (*)
|    |    \--- org.junit.jupiter:junit-jupiter-engine:5.9.3 FOCUS_ON_ME!!!
|    |         +--- org.junit:junit-bom:5.9.3 (*)
|    |         +--- org.junit.platform:junit-platform-engine:1.9.3 FOCUS_ON_ME!!!
|    |         |    +--- org.junit:junit-bom:5.9.3 (*)
|    |         |    +--- org.opentest4j:opentest4j:1.2.0
|    |         |    \--- org.junit.platform:junit-platform-commons:1.9.3 (*)
|    |         \--- org.junit.jupiter:junit-jupiter-api:5.9.3 (*)
|    +--- org.mockito:mockito-core:5.3.1
...

Vampire · February 26, 2024, 3:56pm

The problem is, that you use mavenLocal() in your repositories and as first one and without content filtering.

mavenLocal() is broken by design in Maven already.

And using it in a Gradle build makes the build slower and flaky as you just observed.

You can read more details about it at Declaring repositories.

What happens here is, that on Jenkins some Maven project was built that used com.fasterxml.jackson.core:jackson-core:2.15.0 which put the files Maven needed - and only those files - to mavenLocal() which by Maven is not only used as a repository, but also as a download cache. Now when the Gradle build runs which you told to use mavenLocal() as normal repository, it finds the POM of that version in there, does see the redirection to the Gradle Module Metadata, but does not find the Gradle Module Metadata as Maven of course did not download it to there, so Gradle uses the information in the POM.

But as the Gradle Module Metadata is mightier than the POM format, it has additional information like that dependency, that the POM does not have.

So on the machines where it works, the lib is not in mavenLocal(), so it is used from mavenCentral() or whatever else you declared, where the GMM is available and can be used, but on Jenkins the broken state of the dependency is in mavenLocal() and thus is used, causing this difference in dependency resolution.