How do I include ivy artifacts by pattern

Help/Discuss
1

In Ivy I can have dependency declared like this:

<dependency org="org" name="module_name" rev="12" conf="conf_name->*">
  <include name="foo(.*)-bar" ext="zip" matcher="exactOrRegexp"/>
</dependency>

which will download all matching files.

How can I define similar (regex-based) dependency in Gradle?

2

Reading https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.Configuration.html#org.gradle.api.artifacts.Configuration:allArtifacts, I would say that you probably have to resolve the full dependency, to get all its artifacts and then only filter the ones you want based on a ‘matching{}’ call on the underlying FileCollection.

I might be wrong though, and something more intelligent may be found

3

By default, Gradle will only try to get one file (default one) per module. How do I retrieve full list of artifacts?
And whats more important, how do I plug into dependency resolution process so that all this plays nicely with Gradle caches, etc.?

4

Not sure what you mean by that.
I have ivy modules that contain several artifacts. When I declare a dependency on theses modules, gradle automatically retrieves all their artifacts.

Give me a precise extract of your ivy.xml and the artifacts you want to retrieve. I will then write a working example.

5

My bad, looks like I misread something in documentation.
As to examples, lets say I have the following ivy.xml

<ivy-module version="1.4">
  <info organisation="org" module="bt390" revision="501655:id"/>
  <publications>
    <artifact name="kotlin-android-extensions-plugin-0.12.529" type="zip" ext="zip"/>
    <artifact name="kotlin-bare-plugin-0.12.529" type="zip" ext="zip"/>
    <artifact name="kotlin-compiler-0.12.529" type="zip" ext="zip"/>
    <artifact name="kotlin-compiler-for-maven" type="jar" ext="jar"/>
    <artifact name="kotlin-compiler-javadoc" type="jar" ext="jar"/>
    <artifact name="kotlin-compiler-sources" type="jar" ext="jar"/>
    <artifact name="kotlin-plugin-0.12.529" type="zip" ext="zip"/>
    <artifact name="kotlin-reflect-sources-for-maven" type="jar" ext="jar"/>
    <artifact name="updatePlugins" type="xml" ext="xml"/>
</publications>
</ivy-module>

I my Gradle script I want to declare dependency on all files, that match kotlin-compiler-.*\.jar pattern (all jars with name starting with kotlin-compiler)
How do I do this?

6

Looks like you will not be able to use regexp as simply as in ant (for the moment, please Gradle Devs correct me if I’m wrong)

Several things you can do though:

  • Declare ‘conf’ in your ivy artifacts

artifact name=“kotlin-compiler-0.12.529” type=“zip” ext=“zip” conf=“myConf”
artifact name=“kotlin-compiler-for-maven” type=“jar” ext=“jar” conf=“myConf”

This can then be used to retrieve only the artifacts you want
i.e.

dependencies {
compile group:‘org’ module:‘bt390’ version:‘501655:id’ conf:‘myConf’
}

This supposes you can modify the ivy.xml descriptor file of your bt390 module.

  • Explicitly tell which artifacts you want on the dependency (using this method)

dependencies {
compile ‘org:bt390:501655’ {
artifact {
name = ‘kotlin-compiler-0.12.529’
}
artifact {
name = ‘kotlin-compiler-for-maven’
}
…
}
}

  • Create a configuration, and call an ArtifactQuery on it ?
    I’m not an expert on this, but you might create a ‘fake’ configuration and execute a ArtifactResolutionQuery on it, to filter only your wanted artifacts.
    Sth like

configurations {
kotlincompiler
}
dependencies {
kotlincompiler ‘org:bt390:501655’
}
def componentIds = configurations.kotlincompiler.incoming.resolutionResult.allDependencies.collect { it.selected.id }
def result = dependencies.createArtifactResolutionQuery()
.forComponents(componentIds)
.withArtifacts(not sure what to put here)
.execute()
def filteredFiles = result.resolvedComponents.getArtifacts(not sure what to put here).findAll{it instanceof ResolvedArtifactResult && it.file.name.contains(your regexp)}.collect{it.file}
dependencies {
compile filteredFiles
}

1 Like
7

Thank you for your help. I will try to investigate Artifact Resolution Query API closer.
Unfortunately, I’ll definetly need to get the pattern matching working to start migration of my project to Gradle, because some dependencies are available via Ivy only and have build numbers inside their names…

8

IMHO, using a regexp is not something I would do, because it gives you too much control on the artifact you want to retrieve. It’s not your responsability, but the responsability of the dependency you whant to retrieve, to tell you what artifacts it needs in various cases (at runtime, to compile, under 64 bits environement, etc etc)
That’s the purpose of the Ivy conf attribute.

That being said, you might not have the freedom to change the ivy.xml of your dependency. If your purpose is a full migration to Gradle (and this dependency is something you produce as well) I would consider changing the way it is declared as well.

During the transition phase, why not calling directly the AntBuilder shipped with Gradle, in order to call from Ant the ivy resolve and ivy retrieve tasks ? Never tested but it should work, and at least you could retrieve your dependencies using ant-style pattern, put them in a lib directory, and declare your Gradle dependencies to that lib directory.

9

After some more trial-and-error, I was able to extend Gradle to resolve dependencies with following syntax:

dependencies {
    compile "org:module_name:12" {
        artifact {
            name "foo.*-bar"
            type "zip"
        }
    }
}

For this, one will need project evaluation listener, that will post-process the dependencies. Resolve ivy descriptor for each dependency, parse it, match the artifact names, update the dependency’s artifacts descriptors (remove the one with pattern in name and insert matched artifacts with names).

Pros:

  • Properly uses Gradle’s artifacts cache.
  • Avoids transfer of exra (not matched) artifacts.
  • Dependencies resolution mechanics is applied.

Pitfalls found during implementation:

  • Copy a configuration before resolving ivy descriptors. Resolved configuration (with dependencies) is considered immutable and wont be resolved again, so matched artifacts will not be downloaded

  • Matching different entities. After an Ivy descriptor is “resolved” and downloaded, it is somewhat tricky to match it with unresolved dependency (to update artifact descriptors), as resolved entity has different type. So far, matching on “group-artifact-version” coordinates works, but it is fragile solution.

A sample code for dependency processor can be found on GitHub (disclamer: provided “as is”, no gurantees and responsibilities. But if it blows your project’s working copy, please let me know)