Incorrect Dependency Resolution on binary artifact - Gradle forcing project dependency?


(Ferzan Rizvi) #1

Why does the dependency resolution seem to be taking a project configuration even though it is defined as a binary dependency from a repository ie. group: project.group, name: project.name, version: project.version.toString(), ext: ‘zip’. This build script is apart of the larger gradle build which builds and publishes the artifacts.

It strictly a deployment buildscript for release artifacts used as a script plugin on 2 different packaging projects. I tried changing the dependency resolution caching as well which did not help. Code snippets of one of the packaging projects, deployment plugin, and download plugin files are below.

I’ve tested on 2 different versions of gradle 1.8 and 1.10 but most of the potential fixes were tested on 1.10.

Any help would be appreciated, I’ve been mucking around with this for hours.

-Ferzan

----------------------------------------------------------------- bulk-update-osgi\build.gradle -----------------------------------------------------------------

task archivePkg(type: Zip) {
    baseName = project.name
    into('jcr_root') {
        from("${rootDir}/bulkupdate/src/main/content/jcr_root/") {
            exclude '**/.vlt'
            exclude '**/*.orig'
            exclude '**/.hg'
        }
    }
    into('META-INF') {
        from("${rootDir}/bulkupdate/src/main/content/META-INF")
    }
      from bulkupdateCopySpec
      from(configurations.embedded) {
        into('jcr_root/apps/sni-bulkupdate/install/')
    }
}
  task build(dependsOn: archivePkg)
  apply from: "$rootDir/gradle/publishArtifact.gradle"
artifacts {
    archives archivePkg
}
  apply from: "$rootDir/gradle/deployOsgiBundle.gradle"

----------------------------------------------------------------- downloadOsgiBundle.gradle -----------------------------------------------------------------

apply from: "$rootDir/repositories.gradle"
  configurations {
    deployCq
}
  configurations.deployCq{
    resolutionStrategy{
        cacheDynamicVersionsFor 0, "seconds"
        cacheChangingModulesFor 0, 'seconds'
    }
}
  dependencies {
    if (env != 'LOCAL' && !version.toString().contains('SNAPSHOT')) {
        logger.quiet "Using Release Bundle ${project.group}:${project.name}:${project.version} for Deployment Config"
        deployCq group: project.group, name: project.name, version: project.version.toString(), ext: 'zip'
    }else{
        logger.quiet "Using Snapshot Bundle ${archivePkg.archivePath} for Deployment Config"
        deployCq files(archivePkg.archivePath)
    }
}
  task fetchDeployArchives(type: Copy) {
    println "PATH:" + project.configurations.deployCq.singleFile.canonicalPath
    from project.configurations.deployCq
    into "$buildDir/download/artifacts" as File
}

--------------------------------------------------------------------------------- StdOut ---------------------------------------------------------------------------------

:buildSrc:build UP-TO-DATE
Loading configuration for environment 'DEV'.
Using Release Bundle com.scrippsnetworks.wcm:bulkupdate-osgi:2.2.164 for Deployment Config
PATH:C:\Repos\fnr-imp\bulkupdate-osgi\build\distributions\bulkupdate-osgi-2.2.164.zip
<----- THIS IS THE ISSUE.. it should be resolved from "$buildDir/download/artifacts" after downloading from the repo
Using Release Bundle com.scrippsnetworks.imp.fnr:imp-osgi:2.2.164 for Deployment Config
PATH:C:\Repos\fnr-imp\imp-osgi\build\distributions\imp-osgi-2.2.164.zip
:bulkupdate-osgi:fetchDeployArchives UP-TO-DATE
:bulkupdate-osgi:cqDeploy-Author-10.47.28.211
Deploying artifact 'C:\Repos\fnr-imp\bulkupdate-osgi\build\distributions\bulkupdate-osgi-2.2.164.zip'

(Peter Niederwieser) #2

I don’t understand which problem you are trying to solve. You are copying the resolved artifact into ‘"$buildDir/download/artifacts"’. But why should it originally be resolved from there?

Also note that artifacts resolved from repositories with a ‘file:’ URL aren’t cached. I suppose that’s why the path starts with ‘C:\Repos’ rather than ‘~/.gradle’.


(Ferzan Rizvi) #3

Let’s see if I can explain it a little better. We are using gradle for our Continuous Delivery pipeline in Jenkins, it is modeled after your Continuous Delivery talk and presentation based on the toDo project on GitHub. The first stage is ‘gradle publish’, the src is built, packaged, and uploaded to the repository with a release version ie. 1.0.2 (example project ‘bulk-update-osgi.gradle’).

The next stage is deployment to the lower environments where we would like gradle to download the artifact from the repository (downloadOsgiBundle.gradle) and execute deployment tasks against it. Since it’s a pipeline it’s simple enough to copy the old workspace and use the built artifact from the first stage or just create a project level dependency between build and deploy. However that is not what we are looking for, we want to be able to choose which version to deploy when it goes to the stage and production environments along a few other requirements like being able to deploy artifacts to developer sandboxes on the fly for troubleshooting.

In order to do that without rebuilding from src it makes sense to download the artifact from the repo and then execute deployment. That’s where the issue is occurring, normally when a dependency is defined like in downloadOsgiBundle.gradle, gradle hits the repo, downloads the artifact, and everything is great but, for some reason in this setup, it’s not downloading the artifact. Instead it’s just resolving the configuration to $buildDir/distributions/ and not downloading the artifact from the repo.

Let me know if you need any other information.


(Peter Niederwieser) #4

Can you show the repository declaration(s)?


(Ferzan Rizvi) #5
repositories {
    mavenLocal()
    maven { url "http://XXXXX/archiva/repository/3rd_party/" }
    maven { url "http://XXXX/archiva/repository/releases/" }
    maven { url "http://XXXX/archiva/repository/snapshots/" }
    maven { url "http://XXXX/archiva/repository/internal/" }
    maven { url "http://repo.adobe.com/nexus/content/groups/public" }
    maven { url "http://snapshots.maven.codehaus.org/maven2" }
    maven { url 'http://repo.spring.io/snapshot' }
    mavenCentral()
    maven {
        url "http://XXXXXXX/snilabs/libs-releases"
        credentials {
            username = 'deployer'
            password = 'XXXXXXX'
        }
    }

The artifacts we need should be downloaded from “http://XXXXXXX/snilabs/libs-releases


(Peter Niederwieser) #6

Can you try with the latest Gradle version? If I’m not mistaken, there used to be a bug where an external dependency whose group, name, and version matched those of a local project was resolved as a project dependency, and I think it has meanwhile been fixed.


(Ferzan Rizvi) #7

No luck, I tried deleting all the cache and using Gradle 1.11-rc-1.

That does sound like the issue I am having though. Is there a workaround that might you know of?


(Peter Niederwieser) #8

The closest I could find is: http://forums.gradle.org/gradle/topics/unable_to_resolve_detached_configuration_with_ourself_in_it

However, this is said to be fixed. Will have to investigate further. What happens when you set the dependency version to something other than ‘project.version’ (assuming that other version exists in the repo)? And what happens when you temporarily change ‘project.version’ in ‘fetchDeployArchives.doFirst { … }’? Could try the same with ‘project.group’.


(Ferzan Rizvi) #9

Manually using a version did not work, it still tries to resolve it from $buildDir/distributions.

deployCq group: project.group, name: archivePkg.baseName, version: '2.2.166', ext: 'zip'

The fetchDeployArchives task always shows UP-TO-DATE so the doFirst block changes do not run.

fetchDeployArchives.doFirst {
 project.version = '2.2.165'
 project.group = 'com.asdf'
}

----------------------------- StdOut ----------------------------- :buildSrc:build UP-TO-DATE Loading configuration for environment ‘DEV’. Using Release Bundle com.scrippsnetworks.wcm:bulkupdate-osgi:2.2.166 for Deployment Config PATH:C:\Repos\fnr-imp\bulkupdate-osgi\build\distributions\bulkupdate-osgi-2.2.166.zip Using Release Bundle com.scrippsnetworks.imp.fnr:imp-osgi:2.2.166 for Deployment Config PATH:C:\Repos\fnr-imp\imp-osgi\build\distributions\imp-osgi-2.2.166.zip :bulkupdate:clean UP-TO-DATE :bulkupdate-osgi:clean UP-TO-DATE :common:clean UP-TO-DATE :etl:clean UP-TO-DATE :imp-osgi:clean UP-TO-DATE :ws:clean UP-TO-DATE :bulkupdate-osgi:fetchDeployArchives UP-TO-DATE :bulkupdate-osgi:cqDeploy-Author-10.47.28.211

I’ll take a look at the link and post back an update.


(Ferzan Rizvi) #10

This may help narrow down the issue.

I am applying this download/deploy script plugin on two different projects. If I hardcode the dependency to use one of the project’s artifacts (projectA), it will use $buildDir/distributions/ for itself but resolve correctly when it is executing the fetch for the 2nd project (projectB).

Example:

dependencies {
        deployCq group: 'com.scrippsnetworks.wcm', name: 'bulkupdate-osgi', version: '2.2.166', ext: 'zip'
}

---------- StdOut ----------- :buildSrc:build UP-TO-DATE Loading configuration for environment ‘DEV’. PATH:C:\Repos\fnr-imp\bulkupdate-osgi\build\distributions\bulkupdate-osgi-2.2.166.zip Download http://XXXXXX/snilabs/libs-releases/com/scrippsnetworks/wcm/bulkupdate-osgi/2.2.166/bulkupdate-osgi-2.2.166.pom Relying on packaging to define the extension of the main artifact has been deprecated and is scheduled to be removed in Gradle 2.0 Download http://XXXXX/snilabs/libs-releases/com/scrippsnetworks/wcm/bulkupdate-osgi/2.2.166/bulkupdate-osgi-2.2.166.zip PATH:C:\Users\user.gradle\caches\modules-2\files-2.1\com.scrippsnetworks.wcm\bulkupdate-osgi.2.166c8ca18e9844b7795c6bcdad5463e041e30a7f76\bulkupdate-osgi-2.2.166.zip :bulkupdate-osgi:fetchDeployArchives UP-TO-DATE :imp-osgi:fetchDeployArchives


(Ferzan Rizvi) #11

Publishing the artifact with an artifactId other than the project.name is a workaround for this issue.

Thanks for all your help Peter.


(Peter Niederwieser) #12

Glad you found a workaround. I filed GRADLE-3016 to track this issue.