How to get full list of dependencies and their meta?

Help/Discuss
1

Hello,

I’m trying to manage dependencies in a project built with Gradle using Nix in order to make our build process more unified and fully deterministic. Nix is a package manager for Unix systems that “makes package management reliable and reproducible”, and is configured in a declarative way using a Haskell-like language.

Currently we have a set of very ugly scripts which assemble a list of Gradle project dependencies in a Nix consumable format.

In simple terms it does this:

  1. Collect list of projects using gradle projects
  2. Iterate through the projects and get their dependencies using
    gradle ${project}:buildEnvironment and gradle ${project}:dependencies
  3. Iterate through the dependencies and try to fetch them.
  4. Fetch POM file for each dependency and try to download it’s dependencies

This process generates two files:

The Nix format includes SHAs for JARs and POM files and URLs for downloading them, which allows Nix to fetch them in a deterministic way.

As you can see this process is extremely heavy and takes over 90 minutes. I’m trying to figure out how I could achieve the same in a simpler way.

Is there some Gradle command or set of commands I could use to:

  • Get the full list of dependencies of a project?
  • Get the URL of JAR and POM Gradle would use to download it?
2

I might add that gradle app:dependencies doesn’t give a full list.

And I’ve also tried what was suggested in another thread here:

tasks.register("getDependencies") {
    doLast {
        println project.configurations.compile.allDependencies
    }
}

But all I get is an empty list: []

3

I guess there’s no way to do this. Might be simpler to write my own POM file parser than try to get this from Gradle.

4

Since finding a way to do the same thing with Gradle efficiently seems to be far too difficult I’ve decided to write my own tool in Go for resolving Maven dependencies:

Here’s how it works:

 $ echo commons-io:commons-io:2.4 | ./go-maven-resolver
https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom
https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom
https://repo.maven.apache.org/maven2/org/apache/apache/9/apache-9.pom

It resolves ~500 dependencies in our project into 744 POM URLs in ~5 seconds. Now we can update the Nix store with new Gradle dependencies in around a minute.