Hi all,
What is the best way, if there is any, to create and maintain a (maven|ivy) mirror for all the dependencies used in a project?
For example, let’s say we download dependency A from repository A and dependency B from repository B into the in-house repository C.
Thanks a million
The best way is to use a repository manager like Artifactory or Nexus.
Hi Peter, the problem may be more complex that just installing a repository locally.
Consider this use case. I’m setting up a project with external dependencies picked up from an external Maven repository. I know for sure that this project will then be used in environments where there is no Internet (or extra-net) access at all. So, in these environments, there will be no possibility to download artifacts from any Maven repository, even if this repository is hosted in our LAN (since these environments reside on other isolated LANs on which we have no control a no possibility to install any repository).
One solution could be to install a Maven repository locally on the system where the Gradle project will be used. This however would require to make this repository mirror the external Maven repository for all the JARs needed by the Gradle project. How can I know this in advance? Moreover, the Gradle project would not be portable from one system to another without porting the local Maven repository too.
The solution I’ve eventually implemented is the following: through a property in gradle.properties I control whether I’m in “online mode” or in “offline mode”. In online mode I’m using the external Maven repository to pick up dependencies. When in online mode, I have a task “exportExtLibs” that scans all the configurations of all the projects (including the classpath configuration of the buildscript element) and saves all the artifacts in a folder called “ext-lib” in each project. When in offline mode, Gradle doesn’t use the external Maven repository any more, but rather a collection of flatDir repositories on the “ext-lib” folders exported previously.
In this way, a developer can set up its Gradle projects while working in a connected environment, export the needed JARs into its local file system, than take the whole workspace and go to the disconnected environment to work in offline mode with the previously exported JARs.
The code I’m using to export the JARs is essentially based on what I’m writing here: http://gradle.1045684.n5.nabble.com/collecting-only-external-dependency-files-tp5117615p5680602.html although the final solution is more elaborated, since, as I said, I’m doing this on all the subprojects of my Gradle multiproject and I’m exporting each different configuration in a different subfolder of ext-lib and handling configuration extensions from one another.
However, there are still some drawbacks. First of all, when working with web applications (apply plugin: ‘war’), a post-process of the resulting WAR is needed to resolve JAR conflicts. In fact, two copies of the same JAR coming from different flatDir repositories are considered different artifacts. This means the WAR may contain the same JAR twice if two projects of the multiproject are declaring the same dependency (also because of bug GRADLE-1050). For the same reason, if two projects require the same dependency but with different versions, the resolution of the conflict must be done “by hand” by post-processing the resulting WAR and keeping only the latest version (for instance) of each JAR. I created a Gradle plugin to handle all of these aspects. However, I’ve still the problem that if project P1 requires lib L1 in version V1 and project P2 requires lib L1 in version V2, and if L1-V1 requires some other libraries that L1-V2 does not, those libraries end up in the resulting WAR because the flatDir repository approach loses the POMs and the dependency declarations, so even if my WAR post-processing determins that I have to discard L1-V1, I can’t determine that I also have to discard some transitive dependencies L1-V1 actually depends on (and which are not required by any other JARs I still need to include). This would be solved if I could “mirror” the external Maven repository into a file-based local Maven repository, which I know Gradle supports (as long as you organize a local folder in a Maven repository-like fashion), instead of into a flatDir repository.
However to follow this better approach I would need a way to export not only the binary JAR artifacts of each project dependency, but also its POM, the source/Javadoc artifacts, etc… This is something I’m not able to perform using the current Gradle API. In fact, another severe problem of my solution is that I can’t export the source JARs in the ext-lib folder, so when working in offline mode the developer will have to work without the sources of the external dependencies, unless it gathers them in some other non-automatic way.
So, it would be really great if Gradle could support such scenarios with some built-in facility to export all the information regarding dependencies into a Maven repository based on the local file system.