Incorrect artifact file resolved from cache for buildscript classpath


#1

Recently, we had quite some troubles with one of our multi-project builds.

While digging into the problem, it turned out that we do have wrong files on the buildscript classpath.

buildscript {
 repositories {
  maven { url repositoryBaseURL + '/repo1' }
  maven { url repositoryBaseURL + '/anotherRepo' }
  mavenLocal()
 }
 dependencies {
  classpath "org.apache.bcel:bcel:2.1"
  classpath "c.c.t:myPlugin:2.6.2"
 }
 println "classpath: " + buildscript.configurations.classpath.files
   }
  ..

The script above leads to the following output:

~/.gradle/caches/artifacts-26/filestore/org.apache.bcel/bcel/2.1/jar/eab80d0721bf3057066e09d80e7fa8069fda9c87/bcel-2.1.jar
~/.gradle/caches/artifacts-26/filestore/c.c.a/c.c.a.anotherProject/0.0.0.20131129-085741.5124/jar/bc24ae0a3181c35ea96c7474f6b67e97dad1036/c.c.a.anotherProject-0.0.0.20131129-085741.5124.jar

“c.c.a.anotherProject-0.0.0.20131129-085741.5124.jar” is on the classpath (and has nothing to do with “myPlugin”), but we never specified this one. Additionally, “myPlugin” is missing completely on the classpath.

After cleaning the gradle cache, the build does not fail any longer and we do get the expected files on buildscript classpath. (Note: we are using gradle 1.8)


#2

… happened once again (using gradle 1.9 now). This time the other dependency specified in the buildscript section was missing on the actual buildscript classpath.


(Peter Niederwieser) #3

Hard to say why this would happen. Perhaps “myPlugin” changed without having its version number bumped. One thing I’d try to get rid of is “mavenLocal”. Including this means that every developer’s search path will be different, leading to less consistent build results.


#4

We already got rid of “mavenLocal” right after this problem appeared for the very first time. @your first thought: I’m pretty sure that we pumped the version of the plugin mentioned right before publishing it. Anyway, this time it happened without having a change at all in the buildscript section for a week.


(Peter Niederwieser) #5

Strange. It’s not something I’ve heard of before. If you find out more, let us know.


#6

… happened again (bcel was replaced by protoc on buildscript classpath).

Sadly, I’m still not able to provide an example which shows up the actual problem. But maybe the debug log might give you some hint:

I’m wondering about this line which in fact indicates our problem: [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleVersionRepository] Found artifact ‘org.apache.bcel:bcel:2.1:bcel.jar’ in resolver cache: C:\Users<USERNAME>.gradle\caches\modules-2\files-2.1\com.google.protobuf\protoc\2.5.0\267b2e8ac1b38481e25253da72b4302d10810029\protoc-2.5.0-win32.exe

[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Visiting configuration org.apache.bcel:bcel:2.1(default). [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Visiting configuration c.c.t:myPlugin:2.6.2(default). [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Attaching :building:unspecified(classpath) to its parents. [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Attaching org.apache.bcel:bcel:2.1(default) to its parents. [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Attaching c.c.t:myPlugin:2.6.2(default) to its parents. [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.oldresult.TransientConfigurationResultsBuilder] Flushing resolved configuration data in Binary store in C:\Users<USERNAME>\AppData\Local\Temp\gradle282855522417523883.bin. Wrote root :building:unspecified:classpath. [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache artifact-at-repository.bin (C:\Users<USERNAME>.gradle\caches\modules-2\metadata-2.1\artifact-at-repository.bin) [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleVersionRepository] Found artifact ‘org.apache.bcel:bcel:2.1:bcel.jar’ in resolver cache: C:\Users<USERNAME>.gradle\caches\modules-2\files-2.1\com.google.protobuf\protoc\2.5.0\267b2e8ac1b38481e25253da72b4302d10810029\protoc-2.5.0-win32.exe [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleVersionRepository] Found artifact ‘c.c.t:myPlugin:2.6.2:myPlugin.jar’ in resolver cache: C:\Users<USERNAME>.gradle\caches\modules-2\files-2.1\c.c.t\myPlugin\2.6.2\986b8fd00dac9fcf6c93543eea9f783bdc0a6f18\myPlugin-2.6.2.jar [QUIET] [system.out] classpath: [C:\Users<USERNAME>.gradle\caches\modules-2\files-2.1\com.google.protobuf\protoc\2.5.0\267b2e8ac1b38481e25253da72b4302d10810029\protoc-2.5.0-win32.exe, C:\Users<USERNAME>.gradle\caches\modules-2\files-2.1\c.c.t\myPlugin\2.6.2\986b8fd00dac9fcf6c93543eea9f783bdc0a6f18\myPlugin-2.6.2.jar] [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on no_buildscript class cache for build file ‘C:<PATH_TO_PROJECT>\building\build.gradle’ (C:\Users<USERNAME>.gradle\caches\1.9\scripts\build_7jvf0l4e7uv842s9f9rj8qvpuc\ProjectScript\no_buildscript). [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired.


(Brian McDonald) #7

I just updated the issue with what seems to be the same problem under 2.2.1 and it may be linked to using --refresh-dependencies.

Here’s a simple java project with one dependency

compile - Compile classpath for source set ‘main’. — org.slf4j:slf4j-api:1.7.7

but the classpath is

/home/cruisecontrol/.gradle/caches/modules-2/files-2.1/org.apache/apache/7/6ebc3957064d9ca9d75431eb37ff40b380e64103/apache-7.pom

If I delete the cache and rerun it the classpath correctly point to the slf4j-api-1.7.7.jar in the cache. We’re still trying to track down the trigger, but knowing more about how the cache goes from the dependency to the file would help. Is there a metadata map being used (so it’s possible to go from one group:module:version to a totally different file? If so, how can that be incorrectly updated or corrupted?


#8

Can you reliably reproduce this problem? I’m unable to create a test case that allows me to track it down.

Regarding the cache: the entry point is ‘org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository’, which is backed by a ‘org.gradle.cache.internal.btree.BTreePersistentIndexedCache’. Inspecting the code I can’t see any obvious bug that would cause this issue, and we’d need to schedule some time to investigate further. A reproducible error would greatly expedite this.