Need a Gradle task to copy ALL dependencies to a local maven repo

Help/Discuss
1

Continuing the discussion from Cache Dependencies into Local Maven Repository from Gradle:

Has anyone done this? I need to install ALL dependencies (those for the Android app that I am building and those for the Android build tooling/plugins) into a local Maven repository so I can perform truly offline (no Internet connection whatsoever) builds.

I do not really know where to start!

I tried simply copying the Gradle cache to the offline machine but I could not get it to work and I have been fighting that battle too long.

I need a Gradle task that populates a local maven repo with all the needed dependencies so I can move it over to the offline machine and be able to perform Gradle builds with no Internet connection.

Thanks!

1 Like
2

You might try this plugin.

3

Thanks for the tip!

Ivypot works pretty well. With a little coddling I was able to download all of the dependencies into a local ivy repo.

I do wish it was a local Maven repo (rather than ivy) so I could serve it up with Nexus in our offline environment.

4

Here is an example on how to achieve this with the Artifact Query API.

5

Thanks! I’ll give this a shot…

6

This seems to write the artifacts in the old, flat Maven directory structure so a gradle build fails because it cannot find the artifacts. I will attempt to modify the task to put the jars and poms in the directory structure expected by a standard “repository maven” declaration in gradle.

7

It also only gets the first-level dependencies. I am still working on a version that will do what I need it to do.

8

Have you got further ? Do you please have a project to look at / help with ?

9

Yes. I will post tomorrow.

Harry Rowell
lessharry@gmail.com

10

Here is the python script that I ended up with to copy all Gradle dependencies from the cache into a local Maven repo. The local Maven repo ends up in the “android-tooling-repo” directory (which you can rename as you like).

#!/usr/bin/python

import sys
import os
import subprocess
import glob
import shutil

def main(argv):
    project_dir = os.path.dirname(os.path.realpath(__file__))
    repo_dir = os.path.join(project_dir, "android-tooling-repo")
    temp_home = os.path.join(project_dir, ".gradle_home")
    if not os.path.isdir(temp_home):
        os.makedirs(temp_home)
    
    if os.path.isdir(repo_dir):
        shutil.rmtree(repo_dir)
    
    subprocess.call(["gradle", "-g", temp_home, "-Dbuild.network_access=allow"])
    
    cache_files = os.path.join(temp_home, "caches/modules-*/files-*")
    for cache_dir in glob.glob(cache_files):
        for cache_group_id in os.listdir(cache_dir):
            cache_group_dir = os.path.join(cache_dir, cache_group_id)
            repo_group_dir = os.path.join(repo_dir, cache_group_id.replace('.', '/'))
            for cache_artifact_id in os.listdir(cache_group_dir):
                cache_artifact_dir = os.path.join(cache_group_dir, cache_artifact_id)
                repo_artifact_dir = os.path.join(repo_group_dir, cache_artifact_id)
                for cache_version_id in os.listdir(cache_artifact_dir):
                    cache_version_dir = os.path.join(cache_artifact_dir, cache_version_id)
                    repo_version_dir = os.path.join(repo_artifact_dir, cache_version_id)
                    if not os.path.isdir(repo_version_dir):
                        os.makedirs(repo_version_dir)
                    cache_items = os.path.join(cache_version_dir, "*/*")
                    for cache_item in glob.glob(cache_items):
                        cache_item_name = os.path.basename(cache_item)
                        repo_item_path = os.path.join(repo_version_dir, cache_item_name)
                        print "%s:%s:%s (%s)" % (cache_group_id, cache_artifact_id, cache_version_id, cache_item_name)
                        shutil.copyfile(cache_item, repo_item_path)
    shutil.rmtree(temp_home)
    return 0

if __name__ == "__main__":
    sys.exit(main(sys.argv))
3 Likes
11

Hi,
I am a rather late to this discussion but this may be of use to someone struggling with getting gradle project dependencies into maven local.

A day ago while trying to migrate several related projects from ant with ivy to gradle and got into trouble when a not yet converted top level ivy project that relied on recently gradle-ized dependency libs could no longer build on machines where the maven local cache was not already populated.

The emergency solution I ended up with for a gradle project that uses the maven plugin relies on Maven itself to populate maven local with the dependencies of the gradle project. I’m posting it here in case it is of use to anyone in a pinch:

$ cd my-gradle-project
$ gradle publishToMavenLocal (not needed if the gradle generated pom file is still available.)
$ cd build/publications/[maven|mavenJava]
$ cp pom-default.xml pom.xml
$ mvn dependency:purge-local-repository

The mvn dependency task purge-local-repository will purge maven local of any of the dependencies found in the pom.xml. However, by default it also downloads and reinstalls them all.

Cheers.

12

I also write a python script to clone my dependency into local maven.


You can simply install by pip install syndle

13

I have found the gradle-offline-dependencies-plugin to work well. It is now available in the Gradle plugins repository also.

14

This was incredibly useful. I needed to set up a command line gradle pipeline based off of zips containing another developer’s gradle project and gradle cache. This was exactly what I needed. All I had to do was run your script and then modify the build.gradle to contain a maven { url “file://path/to/maven/repo” } in the repositories section. Thanks for sharing this!

15

For anyone reading this who just wants to get the jar/pom files of the gradle cache into his local maven repo, you can simply use this gradle task:

task cacheToMavenLocal(type: Copy) {
  from new File(gradle.gradleUserHomeDir, 'caches/modules-2/files-2.1')
  into repositories.mavenLocal().url
  eachFile {
    List<String> parts = it.path.split('/')
    it.path = (parts[0]+ '/' + parts[1]).replace('.','/') + '/' + parts[2] + '/' + parts[4]
  }
  includeEmptyDirs false
}

Note that this can take a long time if you have a big cache.

5 Likes
16

@Adrodoc55 your code snippet was almost exactly what I needed, thanks for having posted it!

I’m saying almost because on my side I had to deal with a library named with dots and I had to turn this bit of your code:
(parts[0]+ '/' + parts[1]).replace('.','/')
into this one:
parts[0].replace('.','/') + '/' + parts[1]

Otherwise it would create a wrong path for libraries such as this one taken from my gradle cache:

org.jboss.spec.javax.transaction\jboss-transaction-api_1.1_spec\1.0.1.Final\…\jboss-transaction-api_1.1_spec-1.0.1.Final-sources.jar

5 Likes
17

Hello,

I created my account just to thank you for sharing. I was able to copy my gradle cache to Maven Local repo thanks to your task.

18

This code is wonderful. It’s the perfect solution for downloading Android dependencies to the local repository. Thank you!