Gradle dependency management, and offline mode

Help/Discuss
1

I’m in a catch-22 situation with Gradle. Let me explain:

repo A – only accessible on a VPN.
repo B – only accessible off a VPN, but online.

I would like to be able to:

  • add new dependencies from repo A, when I am on the VPN
  • add new dependencies from repo B, when I am off the VPN, but still online
  • build offline <-- very important

gradle fails if it can’t get to a repo at any stage. So, I add repo A when on VPN, and repo B when off VPN.

Assuming that I don’t need to download any new dependencies, I can use --offline. I check for the offline flag and add both repo A and repo B. This lets gradle use locally cached files from either repo.

But, if I want to add a new dependency from repo B when off the VPN, repo A dependencies fail to resolve.

If I want to add a new dependency from repo A when on the VPN, repo B dependencies fail to resolve.

What I would like, is if gradle could “offline” some repos. So then it could try the local cache for them.

Is this possible?

This is what I’m currently trying to get working:

repositories {
  if (gradle.startParameter.isOffline() || onVPN) {
    maven { url 'https://nexus.mycompanyinternalsite.com/' }
  }
  if (gradle.startParameter.isOffline() || canReachInternet) {
    maven { url 'http://repo1.maven.org/maven2/' }
  }
}
2

You could use my dependency export plugin to export dependencies to a local folder.

When you are disconnected from the remote repository you could use the the exported directory as a repository instead.

Eg:

plugins {
  id "com.lazan.dependency-export" version "0.4"
}
mavenDependencyExport {
   exportDir = file('path/to/local/repo')
} 
repositories {
   if (!remoteRepoAvailable) {
      maven { url file('path/to/local/repo') }   
   } 
}
1 Like
3

Thanks Lance, I’ll give that a go–but this seems a bit crazy. I already have local copies of all the files that I need in the gradle cache. By using this plugin, I will effectively end up with 2 local caches of jars.

It would be better if I could tell gradle to not bother storing local cached jars against the repository that it came from. Instead, use a simple local flat cache.

4

Gradle has its own internal directory layout for storing cached artifacts which, as you’ve pointed out, seems to contain a reference to the repository that it came from.

I think this was a conscious decision by the Gradle team to avoid situations where a build works on one machine (with a dirty cache) but fails on another. These situations are common in the Maven world.

I don’t think it’s possible to change how this works

5

Even at my company we use a local Maven with only dependencies we need.
We have special code to add new dependencies to this local Maven. Remember that the local Maven folder is not equal to the cache. The cache is something which is temp and will not be shared with other developers and build system machines. The code should be buildable and dependencies resolvable even when you delete the cache(test this after renaming the cache folder to something like cache-bkup).

You should have a local repo management project/module with following sequence of repository.
Use the local Repo as the first repository, then JCenter/MavenCentral and then your company specific repo that is only accessible in VPN. The script basically looks up dependencies(also transitive) and stores them into the local maven repo(just the dir with maven folder structure). The ordering will mean that if it is already found in your local repo it will not be downloaded from Internet/VPN repo. So when you run in VPN, if by chance the VPN repo has transitive dependencies on Internet(they may be found in local repo), but this is bad repo structure in itself.

You will definitely need two runs for setting up this local repo one with Internet and other with VPN. If your company repo has dependencies that themselves have transitive dependencies to something only available in Internet then its a bad repo structure. A repo should be able to resolve all transitive dependencies within itself.