Download sources and pom for transitive dependencies


(Willie) #1

I just realized a huge problem with one of my plugins.

In my plugin, i have this

/*
This method is for jvm dependencies.  It allows for a single entry typed in the build file to be
expanded out to include the pom, sources, and javadoc.
 */
project.getDependencies().metaClass.jvm_resolver = { String coords, boolean javadoc = true, boolean sources = true, boolean pom = true ->
    def result = [project.dependencies.create(coords)]
    if (javadoc) result << project.dependencies.create("$coords:javadoc")
    if (sources) result << project.dependencies.create("$coords:sources")
    if (pom) result << project.dependencies.create("$coords@pom")
    result << project.dependencies.create("$coords@jar")
    println("finished building the jar")
    result
}

Which translates to the build like this

dependencies{
    jvm_tap_approved jvm_resolver('org.hibernate:hibernate-testing:5.2.2.Final')

    jvm_tap_testing jvm_resolver('commons-io:commons-io:2.5')
}

Now what that does is it expands out the GAVC code downloads to include the source, pom, and javadoc file in the said dependency. So when this runs:

def jvmDownloadDeps(String config, primary_map){

    // This block establishes the group of primary dependencies
    def primaryDeps = project.configurations.getByName(config).allDependencies as List

    primaryDeps.each { Dependency comp ->
        primary_map[computeComponentID(comp)] = new JVMDependencyArtifact(computeComponentID(comp))
    }

    //ok, this line triggers an event where the POM files will be downloaded
    def deps = project.configurations.getByName(config).incoming.resolutionResult.allDependencies as List
    def artifacts = project.configurations.getByName(config).resolvedConfiguration.resolvedArtifacts as List

    // Between the build.gradle and this code, we now have a list of jvm artifacts that we want to
    // work with.  lets sort them out.
    artifacts.each {
        String componentid = it.id.componentIdentifier.displayName
        if (primary_map.containsKey(componentid)) {
            fillJVMDependencyArtifact(primary_map[componentid] as JVMDependencyArtifact, it)
        } else {
            if (!jvm_trans.containsKey(componentid)) {
                jvm_trans[componentid] = new JVMDependencyArtifact(componentid)
            }
            fillJVMDependencyArtifact(jvm_trans[componentid] as JVMDependencyArtifact, it)
        }
    }
}

It pulls all the artifacts and gives me a series of maps that i can then use to sift thru the rubble and put things where they need. Works great for the primary listed dependencies.

Now the huge oversite on my part. It doesn’t include transient dependencies. Is there a way to expand this code to say also do the same for transient dependencies? I need everything stored to the .m2 folder so that a second job can kick off to push all dependent and transient dependent files to our internal artifactory machine. I can’t do that if i don’t have the pom file…

Now i have been playing and tried changing my BL method to this and removing the logic from the plugin and build file

def jvmDownloadDeps(String config, primary_map){

        // This block establishes the group of primary dependencies
        def primaryDeps = project.configurations.getByName(config).allDependencies as List

        primaryDeps.each { Dependency comp ->
            def compid = computeComponentID(comp)
            
            println(compid)
            
            primary_map[compid] = new JVMDependencyArtifact(compid)

            project.dependencies.with {
                create("$compid:javadoc")
                create("$compid:sources")
                create("$compid@pom")
                create("$compid@jar")
            }
        }

        println(project.configurations.getByName(config).allDependencies as List)

        //ok, this line triggers an event where the POM files will be downloaded
        def deps = project.configurations.getByName(config).incoming.resolutionResult.allDependencies as List
        def artifacts = project.configurations.getByName(config).resolvedConfiguration.resolvedArtifacts as List

        // Between the build.gradle and this code, we now have a list of jvm artifacts that we want to
        // work with.  lets sort them out.
        artifacts.each {
            String componentid = it.id.componentIdentifier.displayName
            if (primary_map.containsKey(componentid)) {
                fillJVMDependencyArtifact(primary_map[componentid] as JVMDependencyArtifact, it)
            } else {
                if (!jvm_trans.containsKey(componentid)) {
                    jvm_trans[componentid] = new JVMDependencyArtifact(componentid)
                }
                fillJVMDependencyArtifact(jvm_trans[componentid] as JVMDependencyArtifact, it)
            }
        }
    }

in the hopes that once i call the resolutionResult method, i can then loop them adding the additional artifact types and then call resolutionResult a second time, but alas, it doesn’t seem that my new create statements are adding the dependencies to the list. and when i try dependencies.add i get an error complaining about a mutation strategy.

HELP


(uklance) #2

To download all artifacts and pom files for a given configuration, see here

To download sources and javadocs see here


(Willie) #3

Hey, this is great stuff, thanks.

Is there any way to get it to download the maven-metadata.xml file? along with giving me the URL that it was downloaded from? Next phase of this is i need to download the last X versions of a library. not just the latest.

Now the next problem is doing an actual publish. I realized last night that my efforts were not creating the maven-metadata.xml file in artifactory, which is kinda important. But i was doing it via the artifactory client. It looks like the gradle client is designed to work with maven central, not artifactory. which is not good for me. I somehow have to figure out a way to get the artifactory client to do its equivalent of

mvn deploy:deploy-file -DpomFile=<path-to-pom> \
  -Dfile=<path-to-file> \
  -DrepositoryId=<id-to-map-on-server-section-of-settings.xml> \
  -Durl=<url-of-the-repository-to-deploy>

thanks