Combining/Resolving Two Repository Types in Gradle

Help/Discuss
1

In my Gradle (2.14) project, I need to define two types of repositories: a flat directory and a Maven repository (a local Nexus server).

I can get both working separately, but can’t get them to play nicely together. Ideally, I would have Gradle look at the local directory first, then at the Maven repository.

Current Setup

I have defined the repositories in my build.gradle like this:

    repositories {
        flatDir dirs: "${rootProject.projectDir}/libs"    
        flatDir dirs: "${rootProject.projectDir}/test.libs"

        maven {
            credentials {
                username nexus_username
                password nexus_password
            }
            url "http://nexus-server/nexus/content/groups/public"
        }
    }

In the libs (and test.libs) directory, the jar file names may or may not have versioning (but when using a flatDir repository, I believe that is irrelevant):

    libs\activation.jar
    libs\imap.jar
    ....
    libs\<closed_source_framework>.jar
    ....
    libs\gson-2.2.4.jar
    libs\stax-utils.jar
    .....

The reason I can’t use our local Nexus server for everything is because of <closed_source_framework>.jar; most of the dependencies in the libs folder come packaged with that distribution, and I can’t reliably get the version information to pull them from Nexus.

Now, one of the other teams is publishing their jars to the Nexus server and I’d like to be able to pull their jars from Nexus, so I have (re)defined my dependencies in my build.gradle:

    dependencies {

        // Grab other-team jar files from Nexus server
        compile "other-team-group:other-team-jar-1:version"
        compile "other-team-group:other-team-jar-2:version"
        compile "other-team-group:other-team-jar-3:version"

        // Grab everything else from 'flatDir'
        compile name: 'activation'
        compile name: 'imap'
        ...
        compile name: 'gson-2.2.4'
        compile name: 'stax-utils'
        .....
        
    }

The Problem

So now comes my problem. I had expected Gradle would search the repositories in the order specified in my build.gradle; meaning that it would look to the local libs folder first and then go to Nexus if it can’t find it locally. What I’m seeing instead is that Gradle is looking at the Nexus server for the jar files already in the local libs folder. Obviously, this is slowing down my build (I have ~30 dependencies defined).

Some Info

Output from gradle properties command, to show repository information:

    .....
    repositories: [org.gradle.api.internal.artifacts.repositories.DefaultFlatDirArtifactRepository_Decorated@18814b1b, org.gradle.api.internal.artifacts.repositories.DefaultMavenArtifactRepository_Decorated@6dff028]
    .....

Output from gradle --info compileJava, to show that Gradle is doing a lookup to Nexus:

    .....
    // Successfully find the other team jar files in Nexus, this is okay
    Download http://nexus-server/nexus/content/groups/public/other-team-group/other-team-jar-1/version/other-team-jar-1-version.pom
    Download http://nexus-server/nexus/content/groups/public/other-team-group/other-team-jar-2/version/other-team-jar-2-version.pom
    Download http://nexus-server/nexus/content/groups/public/other-team-group/other-team-jar-3/version/other-team-jar-3-version.pom
    .....
    // Continues looking in Nexus for jar files that should be found in local libs folder
    Resource missing. [HTTP GET: http://nexus-server/nexus/content/groups/public//activation//activation-.pom]
    Resource missing. [HTTP HEAD: http://nexus-server/nexus/content/groups/public//activation//activation-.jar]
    Resource missing. [HTTP GET: http://nexus-server/nexus/content/groups/public///imap//imap-.pom]
    Resource missing. [HTTP HEAD: http://nexus-server/nexus/content/groups/public//imap//imap-.jar]
    .....

Bottom Line

How can I get Gradle to stop looking at the Maven repository for jar files that I know it will only find locally?

2

Gradle will prefer an artifact with an pom/ivy descriptor over an artifact without. I think this is why gradle continues searching after it finds a match in the flatDir repository. This may or may not solve your problem, but you could use a FileCollectionDependency instead of a ModuleDependency.

Eg:

ext {
   libs = "${rootProject.projectDir}/libs"
   testLibs = "${rootProject.projectDir}/test.libs"
}
dependencies {
   compile files("${libs}/activation.jar", "${libs}/imap.jar")
   compile files("${testLibs}/gson-2.2.4.jar")
   ...
}
1 Like
3

Hi Lance,

Tried a quick mockup of this, and it worked fine. Thank you very much!

Just one comment though - it seems more like a workaround than a solution. It seems like it’s not “the Gradle way”. But either way, I think it will help me.

4

I think the “gradle way” would be to store the files in a maven repository structure, each with a pom descriptor and reference via

repositories {
   maven {
      url uri("${rootProject.projectDir}/mavenRepo")
   }
   maven {
      url uri("${rootProject.projectDir}/mavenTestRepo")
   }
}

A simple way to do this would be using maven command line to install into maven local and then extract the artifacts.

A more common solution would be to install the 3rd party jars to a maven repository (eg nexus or artifactory) hosted somewhere within your company’s firewall and access from there rather than committing jars to source control.

1 Like
5

For the second suggestion, that would be ideal way… and I had considered it. But the third party vendor we license the jars from would not allow us to host it, even inside our own firewall. I might try to lean on them some more…

I’m intrigued by the first method though, I really like it. I will look at that more closely, and see if I can script that in a post-checkout.gradle file that we already have. (We unpack the vendor supplied zip into the libs directory.)

Thanks again Lance, you’ve helped me a lot :+1: