ArtifactResolutionQuery to access parent poms

I’m writing a Gradle plugin which I’m hoping can export dependecies to a local folder for offline use. For this to be useful it needs to export jars, poms and parent poms. It’s currenty working for jars and poms but it’s not getting parent poms from the ArtifactResolutionQuery.

Here’s the code to get the poms

   private void copyPoms(Configuration config) {
        def componentIds = config.incoming.resolutionResult.allDependencies.collect { it.selected.id }

        def result = project.dependencies.createArtifactResolutionQuery()
                .forComponents(componentIds)
                .withArtifacts(MavenModule, MavenPomArtifact)
                .execute()

        for(component in result.resolvedComponents) {
            def componentId = component.id

            if (componentId instanceof ModuleComponentIdentifier) {
                File moduleDir = new File(exportDir, getPath(componentId.group, componentId.module, componentId.version))
                project.mkdir(moduleDir)
                component.getArtifacts(MavenPomArtifact).each { ArtifactResult artifactResult ->
                    File pomFile = artifactResult.file
                    project.copy {
                        from pomFile
                        into moduleDir
                    }
                }
            }
        }
   }

I’ve also tried

def componentIds = config.incoming.artifacts.artifacts.collect { it.id.componentIdentifier }

I have a test failing here

Just bumping this thread as it’s had no response for nearly a week. Hoping a Gradle guru can help

The parent pom is itself yet another component (it has it’s own group/module/version). The only way to do this I expect would be to fetch the POM for a component, parse it’s contents to find the parent, then perform another artifact resolution query on that component, and so on, recursively until you get to the root POM.

Hmm… Hasn’t gradle already done that as part of dependency resolution? Same for boms? Can I request a feature for these to be obtainable via an artifact resolution query similar to normal module poms?

Dependency resolution, yes. But an artifact query isn’t quite the same, we are simply fetching the artifacts for a given component. We are outside the context of a resolved dependency graph at this point. The resolution has already been done by this point but these two things are currently a bit disjointed. You can certainly fill a feature request on Github.

I’ve raised a feature request on github

I was able to workaround this by integrating the Maven Model Builder API with Gradle dependency management.

Commit here for anyone wanting to do similar

2 Likes

Coming back to this (very) old thread, the dependency management docs say

If the module metadata is a POM file that has a parent POM declared, Gradle will recursively attempt to resolve each of the parent modules for the POM.

If that really was the case, I guess this topic would be solved. However, the described behavior does not match what I’m observing in tests: Parent POMs do not seem to be resolved / downloaded to the Gradle cache. At least not as POM files. Are they maybe only serialized in the form of descriptor.bin files?

If that really was the case

It IS the case.
If you only delete <GRADLE_USER_HOME>/caches/modules-2/files-2.1/asm and resolve asm:asm:3.1, then neither its pom, nor the parent pom is downloaded but only the jar, probably because it is stored in the metadata folders as you assumed.
But if you also delete <GRADLE_USER_HOME>/caches/modules-2/metadata-*/descriptors/asm and do the resolution again, then you see that pom, parent pom and jar are downloaded to the cache.

I guess this topic would be solved.

No, has not really much to do with the topic here imho.

I was reading “resolve” in “Gradle will recursively attempt to resolve each of the parent modules for the POM” so that Gradle would download the parent POM and store it as a *.pom file in the cache. That would have solved the topic.

It does resolve the parent pom.
It does put it in the cache. (Except if it has it already in the binary metadata)
Still, this has nothing to do with this thread here and does not solve it in any way.

The “except” part is exactly the point. The bigger topic of @Lance’s plugin is to make (also parent) POMs available offline. I.e. the plugin needs to get parent POM files some somewhere, not necessarily from an ArtifactResolutionQuery (though I agree that would be a convenient solution).

If a regular ArtifactResolutionQuery would always download the parent POMs also as files, even if binary metadata is already available, and put them into Gradle’s cache, the plugin could simply pick them up from the cache. That would solve the original problem of the plugin; too strictly only looking at ArtifactResolutionQuery doing the work directly is already going too much into the space of another way to solve the original problem.

The “except” part is exactly the point. The bigger topic of @Lance’s plugin is to make (also parent) POMs available offline. I.e. the plugin needs to get parent POM files some somewhere , not necessarily from an ArtifactResolutionQuery (though I agree that would be a convenient solution).

As I also answered in Slack, you can perfectly fine get the ancestor POMs through ArtifactResolutionQuerys. And after you got them through it, they should also be in the cache, while it should be totally irrelevant whether they are in the cache directory as files or not if you get them properly through the query.

If a regular ArtifactResolutionQuery would always download the parent POMs also as files , even if binary metadata is already available, and put them into Gradle’s cache, the plugin could simply pick them up from the cache.

Which would be a very bad idea and not very portable across Gradle versions. Cache layout can change and so on. Especially if there is a proper way to do it already, just not convenient.

Gradle is excellent in avoiding unnecessary work. And downloading the parent POM as file and putting it to the cache is totally unnecessary work if it is already present in a more efficient format, so why should Gradle waste time?

If you want the actual POM, there is a way to get it and that is an ArtifactResolutionQuery.

As an example here is how I check for jdk-version activated profiles in dependencies which are not supported by Gradle and thus need to be taken account for manually by component metadata rules to fixup the dependencies:

val checkForJdkActivatedProfiles by tasks.registering {
    doLast {
        val pomReader = MavenXpp3Reader()

        fun collectAncestorPoms(pom: ResolvedArtifactResult, result: MutableList<ResolvedArtifactResult>) {
            val parent = FileReader(pom.file)
                .use { pomReader.read(it) }
                .parent
            if (parent != null) {
                val parentPom = dependencies
                    .createArtifactResolutionQuery()
                    .forModule(parent.groupId, parent.artifactId, parent.version)
                    .withArtifacts(MavenModule::class.java, MavenPomArtifact::class.java)
                    .execute()
                    .resolvedComponents
                    .asSequence()
                    .flatMap { it.getArtifacts(MavenPomArtifact::class.java).asSequence() }
                    .filterIsInstance<ResolvedArtifactResult>()
                    .single()
                result.add(parentPom)
                collectAncestorPoms(parentPom, result)
            }
        }

        dependencies
            .createArtifactResolutionQuery()
            .forComponents(configurations.filter { it.isCanBeResolved }.flatMap { configuration ->
                configuration.incoming.resolutionResult.allDependencies.mapNotNull {
                    (it as? ResolvedDependencyResult)?.selected?.id
                }
            }.distinct())
            .withArtifacts(MavenModule::class.java, MavenPomArtifact::class.java)
            .execute()
            .resolvedComponents
            .asSequence()
            .flatMap { it.getArtifacts(MavenPomArtifact::class.java).asSequence() }
            .filterIsInstance<ResolvedArtifactResult>()
            .map { pom ->
                val ancestorPoms = mutableListOf<ResolvedArtifactResult>()
                collectAncestorPoms(pom, ancestorPoms)
                Pair(pom, ancestorPoms)
            }
            .forEach { (bottomPom, ancestorPoms) ->
                sequenceOf(bottomPom, *ancestorPoms.toTypedArray()).forEach { pom ->
                    FileReader(pom.file)
                        .use { pomReader.read(it) }
                        .profiles
                        .forEach { profile ->
                            if ((profile.activation?.jdk != null) && (!profile.dependencies.isNullOrEmpty())) {
                                println("jdk activated Maven profile with dependencies found in ${pom.id}")
                                if (pom != bottomPom) {
                                    println("ancestor of dependency ${bottomPom.id}")
                                }
                            }
                        }
                }
            }
    }
}

Even if the parent POMs would always land in the cache directory, the logic would not change much as you would still need to parse the POM to get the parent coordinates to get the POM from the cache and so on, so you can right away use the proper supported way to get it.

The point of this thread is to save this boilerplate and instead just be able to also get the ancestor POMs from the initial artifact resolution query.

1 Like

Thanks @Vampire. For anyone interested, my implementation is at ort/OrtModelBuilder.kt at main · oss-review-toolkit/ort · GitHub.