Exploding an archive in a CopySpec

Help/Discuss
1

I’ve been stumped on this one for quite some time.

I’m currently generating a CopySpec from a method. I could not figure out how to “safely” replicate the war plugin behavior of providedCompile. What we’ve been doing until now is something like:

def appCopySpec(Project app, distPath, String confPath, String filterFile) {
  copySpec {
    def appDir = app.projectDir
    def appName = app.name
    // app configuration
    into(confPath) {
      from("$appDir/src/main/etc") {
        def filterProps = new Properties()
        file("$appDir/src/main/dist/${filterFile}.properties").withInputStream {
          filterProps.load(it)
        }
        filter(ReplaceTokens, tokens: filterProps)
      }
    }

    into(distPath) {
      // app jars
      into("apps/$appName/lib") {
        from app.tasks.withType(Jar) // app artifact
        from app.configurations.runtimeClasspath - app.configurations.providedCompile
      }
...
}

Most notably the last block is what triggers the following warning:

The configuration was resolved without accessing the project in a safe manner. This may happen when a configuration is resolved from a thread not managed by Gradle or from a different project. See Troubleshooting Dependency Resolution for more details. This behaviour has been deprecated and is scheduled to be removed in Gradle 6.0

I’ve read Sharing outputs between projects and whatever posts online I could find. I’ve set it up (as far as I can tell) like in the docs.

After defining a new artifact in each subproject ‘tarball’ from the distribution plugin’s distTar task, and replacing the block:

into("apps/$appName/lib") {
  from app.tasks.withType(Jar) // app artifact
  from app.configurations.runtimeClasspath - app.configurations.providedCompile
}

with:

into("apps/$appName")
from(configurations.getByName(appName))

This will correctly track the dependency and trigger the distTar task needed to generate the artifact. However, I’m just using the tar to facilitate packaging of a war-like archive (in that its just an archive of runtimeClasspath minus providedCompile, but without all the WEB-INF, etc, etc.)

Changing this to use zipTree on singleFile will correctly unpack the archives and create the distribution I’m looking for. However it doesn’t seem to track the dependency correctly, even with an explicit builtBy declaration, and as such if the tar file wasnt already generated, it Gradle bails out because it cant find the archive. This can be confirmed by inspecting the taskTree in both cases.

Execution failed for task ‘:installDist’.

Could not read ‘/path/to/subproject/build/distributions/subproject-version.tar’ as it does not exist.

when using the similar:

from(tarTree(configurations.getByName(appName).singleFile))

You might find some similar posts from about a year ago on this topic, as we still haven’t figured it out. Any help is very much appreciated!

2

I’m guessing that ProjectA is causing a Configuration to resolve() which is owned by ProjectB. You haven’t shown how appCopySpec(...) is being called which is likely to show the issue.

A possible solution to this is to delay the resolving until later by using Project.copy(...) instead of a Copy task.

Eg: Instead of

task myCopy(type: Copy) {
   def pb = project(':projectB')
   from pb.configurations.runtime - pb.configurations.provided
   into 'someDir' 
}

You could do

task myCopy {
   ext.pb = project(':projectB')
   inputs.files pb.configurations.runtime
   inputs.files pb.configurations.provided
   outputs.dir 'someDir' 
   doLast {
      copy {
         from pb.configurations.runtime - pb.configurations.provided
         into 'someDir' 
      } 
   } 
}

Or you could move the task definition inside of ProjectB to stop ProjectA from resolving the configurations

3 
distributions {
  main {
    baseName = "my-app"
    contents {
      with generateDistCopySpec()
    }
  }
}

which calls

generateDistCopySpec(String distPath = '', String confPath = 'etc') {
  copySpec {
    into(distPath) {
      // app lib
      into('app-lib') {
        from configurations.appLib
      }
      ....
      // apps
    apps.collect { with appCopySpec(it, distPath, confPath, filterFile) }
  }

This definition again, avoids the warning but does not create the proper task dependency linkage to trigger the distTar task in subprojects if I’m using the tarTree() call to unpack the archive, causing a file not found when the distribution plugin tries to package the root project. Without the tarTree() in the copyspec, it’s resolved correctly and the distTar task of subprojects runs correctly.
subprojects have the artifact i’m consuming defined in the root buildscript in a subprojects {} block

configurations {
    providedCompile
    tarball
  }

  distributions {
    main {
      contents {
        exclude "*.properties"
        from configurations.runtimeClasspath - configurations.providedCompile
        from tasks.withType(Jar)
        into "lib"
      }
    }
  }

  artifacts {
    tarball(distTar)
  }

  sourceSets {
    main.compileClasspath += configurations.providedCompile
  }

By declaring a configuration per “app” (subproject) with a single depenency on the subproject tarball artifact i want to unpack, i am able to avoid that warning, the issue being that if I use from(tarTree(configurations.getByName(…).singleFile)) form (which is a configuration defined in the root project, not cross project, the dependency on the distTar task is not calculated by Gradle. Using from(configurations.getByName(…)) does set up the dependency relationship correctly but that will just pull in the archive without unpacking it.

Rather than using Copy tasks in subprojects, i’m relying on the distribution plugin to generate the distribution archives. Are you suggesting I use copy directly?

4

Interestingly enough, I stopped using the distribution plugin and custom configuraiton and artifacts in the subprojects and simply defined a copy task:

task dist(type: Copy) {
    from configurations.runtimeClasspath - configurations.providedCompile
    from tasks.withType(Jar)
    into file("$buildDir/dist")
  }

Then from the root project consume it with the following where app is the subproject and an instance of Project

from(app.tasks.dist)
into("apps/$appName/lib")

This doesn’t seem to produce any warning and produces the expected result, but I thought it was bad practice to reach into a subproject’s tasks?