Copying the Gradle cache to another machine

Old Forum Archive
1

My company has a contractual obligation to produce an escrow build for every release we make. The acceptance criteria is that the build can be executed on a different PC with no network connectivity and all tools, source and dependencies are self-contained.

When we used ANT and Ivy I just copied the local Ivy cache and the build had a flag to avoid Ivy trying to resolve against our dependency repo (Artifactory) - this meant it only resolved against the local cache. This was simple and we had the same Ant script for both the normal and escrow build.

It’s not obvious to me if the fairly recent caching changes to Gradle would be portable to another user/PC to use the same pattern for a Gradle-based escrow build.

Any other suggestions on how to do this?

2

I haven’t tried, but I think that copying over the Gradle cache like you previously did for the Ivy cache should work fine. If you hit any problems, let us know.

3

As long as you’re not concerned about optimising storage, you should be able to copy the entire ~/.gradle directory, which will include any gradle wrapper downloads as well.

You can then execute your build with the ‘–offline’ switch to force Gradle to use only cached dependencies, even in the case of changing modules (eg SNAPSHOT).

4

I’ve tested my escrow build locally on my PC with the --offline setting and it works, but it fails on another PC.

To make sure I’m using the gradle user home that I think I am, I’ve tested it locally after deleting the cache and the build fails as expected. When the cache is present it works.

The error is:

FAILURE: Build failed with an exception.

  • What went wrong: Could not resolve all dependencies for configuration ‘:classpath’. > Could not download artifact ‘org.jfrog.buildinfo:build-info-extractor-gradle:2 .0.10@jar’: No cached version available for offline mode

  • Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Navigating the Gradle cache I can see that the JAR is present.

This is on Windows 7. I’ve tried running the build from an admin command line and that doesn’t make any difference, so I don’t think it’s a permission issue. I’m logged onto the other PC with the same domain account, so I don’t think it’s a user issue either.

5

Can you please provide a minimal project that reproduces the behaviour?

At the very least we’ll need to see the ‘repositories’ and ‘dependencies’ sections of your build; particularly those used to locate ‘org.jfrog.buildinfo:build-info-gradle-extractor’, the failing dependency.

6

Forget about the jfrog (Artifactory) dependency as I can reproduce the problem with a minimal Java project.

One interesting thing I found was if I run the build on another PC without using the --offline flag, the build succeeds and it does not re-download the dependencies. From that point onward the --offline flag works.

I noticed the artifact-at-repository.bin and artifact-at-url.bin files change (they increase in size) when running the build without the --offline switch.

I used the following build.gradle:

apply plugin: ‘java’

apply plugin: ‘eclipse’

repositories {

mavenCentral()

}

dependencies {

compile group: ‘com.google.guava’, name: ‘guava’, version: ‘12.0’

}

task wrapper(type: Wrapper) {

gradleVersion = ‘1.0’

}

With a main class of:

package ch.hedgesphere.escrow;

import com.google.common.base.Optional;

public class Escrow {

/**

  • @param args

*/

public static void main(String[] args) {

// does nothing except try and use an external dependency

Optional option = Optional.absent();

System.out.println("optional value: " + option.or(Boolean.TRUE));

}

}

Did the build with:

gradlew.bat --gradle-user-home ./gradleHome build

Then zipped and copied the directory to another PC and tried to run it again with:

gradlew.bat --gradle-user-home ./gradleHome --offline build

7

Thanks for the simple example.

I’m guessing that ‘gradleHome’ on the second PC doesn’t have exactly the same path as the first PC? In that case you’re hitting a limitation on the portability of our cache: all cached files are referenced by absolute path.

When you run the build without the --offline flag, Gradle is still performing HTTP requests to download the checksum files for your dependencies (run the build with ‘–debug’ and you’ll see). Gradle then uses artifact reuse to avoid downloading the actual dependency. This is why the build doesn’t work with the ‘–offline’ flag.

Your best bet is to configure the second PC with your Gradle home under exactly the same path on the second machine. I haven’t tested this out, but I definitely experience the behaviour you describe by moving the Gradle home directory.

8

Using the same absolute path for the Gradle cache solves the problem.

This is good enough for me and mandating a particular platform and directory layout for the escrow build are perfectly acceptable.

I’m assuming the the cache is portable across different versions of Windows (while keeping the path the same)?

9

Sorry for intercepting this old thread.

Is there a way to make this work with relative paths as well?

My use case: I want to use a prepared gradle build in a training class distributed to students on USB keys. The training’s location might not have internet connection. I cannot rely on identical absolute paths here since the student’s laptops are all different (Win, Mac, Linux, …)

10

Is there a way to make this work with relative paths as well?

As far as I know, there isn’t.

I want to use a prepared gradle build in a training class distributed to students on USB keys.

You could use a ‘flatDir’ repo (but then you lose transitive dependency management), somehow assemble a file-based Maven repo (perhaps by exporting it from a binary repository manager, or by populating an initially empty local Maven repo using Maven) and distribute that, or have students connect to a binary repository on your machine.

11

Peter, thanks for your quick answer. I guess the easiest way to copy the dependencies from the gradle to a Maven’s pom.xml that populates the local maven repo and let the gradle build use this as repo - just as you’ve sketched.

Would it make sense to add the portability of gradle caches as a feature request? Being force to use Maven as a workaround for this is kind of a pita.

12

This is GRADLE-2690.

13

Any chance that this request will get attention and resources in the near feature?

14

I have the similar problem to steven.dick, however I didn’t understand the solution. Project configuration files which are well executable on PC (it will be PC-1) with internet access. (By the way PC has to have internet access and configured VPN in order to have posibillity download the libs from external repository). So, I copied cache files from PC-1 to PC-2(without internet access) and lanched gradle in offline mode, but I got this error: --------------------------------------------------------------------------------------------------------------------------------------------------

FAILURE: Build failed with an exception.

  • What went wrong: A problem occurred configuring root project ‘MyProject’. > Could not resolve all dependencies for configuration ‘:classpath’.

Could not download artifact ‘org.jfrog.buildinfo:build-info-extractor-gradle:2.0.9:build-info-extractor-gradle.jar’

Could not download artifact ‘org.jfrog.buildinfo:build-info-extractor-gradle:2.0.9:build-info-extractor-gradle.jar’:

No cached version available for offline mode

  • Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 2.305 secs

-------------------------------------------------------------------------------------------------------------------------------------------------- I checked ~/.gradle dir and saw that build-info-extractor-gradle.jar file exist. What I have to do in order to resolve this exeption?

And my configuration files in MyProject folder are:

  • build.gradle:
apply plugin: "java" // enables compile configuration
  apply from: 'project.gradle'
  buildscript {
 repositories {
  mavenLocal()
  maven {
   url 'http://external-server.com:8081/artifactory/plugins-release'
   credentials {
    username = "${artifactory_user}"
    password = "${artifactory_password}"
   }
  }
     }
 dependencies {
  classpath(group: 'org.jfrog.buildinfo', name: 'build-info-extractor-gradle', version: '2.0.9')
 }
}
  allprojects {
 apply plugin: 'artifactory'
}
  artifactory {
 contextUrl = "${artifactory_contextUrl}"
 //The base Artifactory URL if not overridden by the publisher/resolver
 publish {
  repository {
   repoKey = 'libs-release-local'
   username = "${artifactory_user}"
   password = "${artifactory_password}"
   maven = true
       }
 }
 resolve {
  repository {
   repoKey = 'libs-release'
   username = "${artifactory_user}"
   password = "${artifactory_password}"
   maven = true
       }
 }
}
  • project.gradle
defaultTasks 'installMyProjectDependencies'
  def MyProjectPackages = [
 [name: "merchandiseStoreFront",
 target: "MyProject/bin/custom/merchandise/merchandisestorefront/lib",
 dependencies: [
 "net.sf.ezmorph:ezmorph:1.0.6",
 "org.apache.httpcomponents:httpclient:4.1.3",
 "org.apache.httpcomponents:httpcore:4.1.4",
 "org.codehaus.jackson:jackson-core-asl:1.9.11",
 "org.codehaus.jackson:jackson-mapper-asl:1.9.11",
 "jaxen:jaxen:1.1.3",
 "org.json:json:20080701",
 "net.sf.json-lib:json-lib:2.4:jdk13",
 "org.seleniumhq.selenium:selenium-android-driver:2.16.1",
 "org.seleniumhq.selenium:selenium-api:2.16.1",
 "org.seleniumhq.selenium:selenium-iphone-driver:2.16.1",
 "org.seleniumhq.selenium:selenium-remote-driver:2.16.1"
 ]],
 [name: "merchandiseStoreFrontWeb",
 target: "MyProject/bin/custom/merchandise/merchandisestorefront/web/webroot/WEB-INF/lib",
 dependencies: [
 "com.granule:granule:1.0.9",
 "atg.taglib:json-taglib:0.4.1",
 "org.jsoup:jsoup:1.7.2",
 "javax.servlet.jsp.jstl:jstl-api:1.2",
 "org.glassfish.web:jstl-impl:1.2",
 "org.springframework.security:spring-security-taglibs:3.1.1.RELEASE",
 "org.springframework:spring-webmvc:3.1.1.RELEASE"
 ]],
 [name: "webServices",
 target: "MyProject/bin/custom/webservices/lib",
 dependencies: [
 "net.sf.kxml:kxml2:2.1.8",
 "xpp3:xpp3:1.1.4c",
 "oro:oro:2.0.8",
 "org.jsoup:jsoup:1.7.2",
 "org.codehaus.groovy.modules.http-builder:http-builder:0.6",
 "org.apache.httpcomponents:httpclient:4.2.1",
 "org.apache.httpcomponents:httpcore:4.2.1",
 "net.sf.json-lib:json-lib:2.3:jdk15@jar",
 "xml-resolver:xml-resolver:1.2",
 "net.sourceforge.nekohtml:nekohtml:1.9.16",
 "xml-resolver:xml-resolver:1.2"
 ]],
 [name: "webServicesWeb",
 target: "MyProject/bin/custom/webservices/web/webroot/WEB-INF/lib",
 dependencies: [
 "org.codehaus.jettison:jettison:1.3",
 "javax.ws.rs:jsr311-api:1.1.1",
 "javax.servlet:jstl:1.2",
 "org.springframework.security.oauth:spring-security-oauth2:1.0.1.RELEASE",
 "org.springframework.security:spring-security-taglibs:3.1.1.RELEASE",
 "org.springframework:spring-webmvc:3.1.1.RELEASE",
 "com.thetransactioncompany:cors-filter:1.3.2",
 "com.thetransactioncompany:java-property-utils:1.9",
 "javax.xml.stream:stax-api:1.0-2",
 ]],
 [name: "ngrestservices",
 target: "MyProject/bin/custom/ngrestservices/web/webroot/WEB-INF/lib",
 dependencies: [
 "javax.servlet:jstl:1.2",
 "com.fasterxml.jackson.core:jackson-core:2.1.1",
 "com.fasterxml.jackson.core:jackson-databind:2.1.1",
 "com.fasterxml.jackson.core:jackson-annotations:2.1.1",
 "org.springframework:spring-webmvc:3.1.1.RELEASE",
 "org.apache.commons:commons-lang3:3.1",
 "org.codehaus.groovy.modules.http-builder:http-builder:0.6",
 "net.sf.json-lib:json-lib:2.3:jdk15@jar",
 "org.apache.httpcomponents:httpclient:4.2.1",
 "org.apache.httpcomponents:httpcore:4.2.1",
 "xml-resolver:xml-resolver:1.2",
 //"xerces:xercesImpl:[2.6.2,)",
 "net.sourceforge.nekohtml:nekohtml:1.9.16",
 "org.springframework.security.oauth:spring-security-oauth2:1.0.1.RELEASE",
 "org.springframework.security:spring-security-taglibs:3.1.1.RELEASE",
        "com.nastygal:nastycommon:0.0.10"
 ]],
 [name: "ngfulfilmentprocess",
 target: "MyProject/bin/custom/ngfulfilmentprocess/lib",
 dependencies: [
 "org.apache.httpcomponents:httpclient:4.2.1",
 "org.apache.httpcomponents:httpcore:4.2.1",
        "com.nastygal:nastycommon:0.0.10"
 ]],
 [name: "merchandiseCore",
 target: "MyProject/bin/custom/merchandise/merchandisecore/lib",
 dependencies: [
 "net.sourceforge.cssparser:cssparser:0.9.11",
 "net.sourceforge.htmlunit:htmlunit:2.13",
 "net.sourceforge.htmlunit:htmlunit-core-js:2.13",
 "net.sourceforge.nekohtml:nekohtml:1.9.19",
 "org.apache.commons:commons-lang3:3.1",
 "org.apache.commons:commons-exec:1.1",
 "org.apache.httpcomponents:httpclient:4.3.1",
 "org.apache.httpcomponents:httpcore:4.3",
 "org.apache.httpcomponents:httpmime:4.1.3",
    "org.json:json:20131018",
 "org.seleniumhq.selenium:selenium-firefox-driver:2.39.0",
 "org.seleniumhq.selenium:selenium-api:2.39.0",
 "org.seleniumhq.selenium:selenium-remote-driver:2.39.0",
 "org.seleniumhq.selenium:selenium-support:2.39.0",
 "xerces:xercesImpl:2.11.0"
   ]],
 [name: "ngemail",
 target: "MyProject/bin/custom/ngemail/lib",
 dependencies: [
 "org.apache.httpcomponents:httpclient:4.2.1",
 "org.apache.httpcomponents:httpcore:4.2.1"
         ]],
 [name: "ngacceleratorservices",
 target: "MyProject/bin/custom/ngacceleratorservices/lib",
 dependencies: [
 "org.apache.commons:commons-lang3:3.1",
 "com.paypal.sdk:paypal-core:1.4.1",
 "com.paypal.sdk:merchantsdk:2.3.101",
 "com.google.code.gson:gson:2.2.2"
 ]],
 [name: "braintree",
 target: "MyProject/bin/custom/braintree/lib",
 dependencies: [
 "com.braintreepayments.gateway:braintree-java:2.29.0"
 ]]
]
  configurations {
 MyProjectPackages.each { pkg ->
  add(pkg.name)
  configurations[pkg.name].transitive = false
 }
}
  /*
  * Dependencies are downloaded to...
 * /Users/${your_username}/.gradle/caches/artifacts-15/filestore
 */
dependencies {
 MyProjectPackages.each { pkg ->
   pkg.dependencies.each { dep ->
    add(pkg.name, dep)
  }
 }
}
  /*
  * Create copy task for each MyProject package. These will be
 * declared as dependencies for the installMyProjectDependencies task.
 */
MyProjectPackages.each { pkg ->
 tasks.add(
  task(pkg.name, type: Copy) {
   from(configurations[pkg.name])
   into(pkg.target)
  }
 )
}
  /*
 * gradle installMyProjectDependencies
 *
 * Use this task to initialize a new MyProject installation with the required jars
 * for all custom extensions. The jars for the MyProject platform and base
  * extensions should be installed from the zip file downloaded from the MyProject
 * site.
Additionally, in order to keep dependencies upto date and free of
  * stale/unused artifacts, this tasks depends on cleanMyProjectDependencies.
  */
task installMyProjectDependencies(
  dependsOn:
['a_cleanMyProjectDependencies'] +
      MyProjectPackages.collect{ pkg -> pkg.name }
) << {}
  /*
 * gradle cleanMyProjectDependencies
 *
 * Deletes all jar files from the configured MyProjectPackages.
 */
task a_cleanMyProjectDependencies << {
 // The Gradle Delete task does not handle wildcards, so I am using antbuilder.
 // See http://gradle.1045684.n5.nabble.com/Delete-with-wildcard-td4550586.html.
 ant.delete(includeEmptyDirs: 'false', quiet: 'true') {
  MyProjectPackages.each { pkg ->
    fileset(dir: file(pkg.target), includes: '*.jar')
  }
 }
}

Conf file gradle.properties in ~/.gradle folder contains:

artifactory_user=my_name
artifactory_password=my_password
artifactory_contextUrl=http://external-repository.com:8081/artifactory

And system variable in PC-1 and PC-2 are the same:

D:\growthHub\apache-maven-3.1.1\bin;
D:\growthHub\gradle-1.11\bin;
D:\growthHub\apache-ant-1.9.3\bin;
15

I have found a solution. A few files in C:\Users"USER_NAME".gradle folder contains paths to libs with <USER_NAME>. I tried to replace “USER_NAME” to my user name, however it wasn’t succesfull. I decided change path to .gradle folder from C:\Users<USER_NAME>.gradle to D:.gradle, download libs on PC-1 and then move .gradle folder from PC-1 to PC-2. On PC-2 I launch folowing command:

gradle myTask --gradle-user-home D:\.gradle --offline

and it worked. In order to change gradle folder on PC-1 you have to use the following command:

gradle --gradle-user-home D:\.gradle --offline
16

On a related note, it there an elegant way to pre-populate an (empty) Gradle cache with the required dependencies without building the app first? This suggestion does not work as it throws:

> No signature of method: org.gradle.api.internal.artifacts.dsl.dependencies.Def
aultDependencyHandler_Decorated.dependsOn() is applicable for argument types: (j
ava.util.ArrayList) values: [[org.gradle.api.internal.artifacts.dsl.dependencies
.DefaultDependencyHandler_Decorated@56cda38b, ...]]
17

Nevermind, Peter already showed how to do this over here.