Sync task appears to process the same dependency multiple times

Help/Discuss
1

I’m having an apparent issue with a Sync task where it logs (and presumably unzips) each file four times rather than the expected once. The main parts of the cut-down build.gradle I created are below; the real one contains 15+ dependencies and takes over 15 minutes

configurations {
  thirdparty
}

dependencies {
  thirdparty 'com.github:lib_jansson:2.11'
}

task syncThirdPartyFolders(type: Sync) {
  from {
    configurations.thirdparty.collect {
      println("File: " + it)
      zipTree(it)
    }
  }
  into "ThirdParty"
}

The console output:

C:\test\TestingGradleSync>gradle syncThirdPartyFolders
NDK is missing a "platforms" directory.
If you are using NDK, verify the ndk.dir is set to a valid NDK directory.  It is currently set to C:\ExternalTools\android_sdk_r24\ndk-bundle.
If you are not using NDK, unset the NDK variable from ANDROID_NDK_HOME or local.properties to remove this warning.

Incremental java compilation is an incubating feature.
File: C:\Users\me\.gradle\caches\modules-2\files-2.1\com.github\lib_jansson\2.11\e46ccdc64bc97102df2eb55ae0639d2c1a879091\lib_jansson-2.11.zip
:syncThirdPartyFolders
File: C:\Users\me\.gradle\caches\modules-2\files-2.1\com.github\lib_jansson\2.11\e46ccdc64bc97102df2eb55ae0639d2c1a879091\lib_jansson-2.11.zip
File: C:\Users\me\.gradle\caches\modules-2\files-2.1\com.github\lib_jansson\2.11\e46ccdc64bc97102df2eb55ae0639d2c1a879091\lib_jansson-2.11.zip
File: C:\Users\me\.gradle\caches\modules-2\files-2.1\com.github\lib_jansson\2.11\e46ccdc64bc97102df2eb55ae0639d2c1a879091\lib_jansson-2.11.zip

BUILD SUCCESSFUL

Total time: 9.75 secs

I’m curious as to whether this is some gross misuse of Sync tasks, and/or whether there’s a straightforward solution to get this task to process each dependency only once.

2

Gradle is smart enough to unzip an archive at most once. Gradle will unzip to a temp location and copy from there. The temp directory includes a hash of the zip in the path

You have used the Sync.from(Closure) method which wraps your Closure in a FileCollection via Project.files(Object). Each time the FileCollection is iterated, your closure will fire

As I expected there’s one println before task execution which is probably for the up to date check. The other three during and task execution. Sync needs to compare files in two locations to see what’s changed so the extra iterations are probably related to the extra checks.

Print out the stack trace in your closure if you’re interested

The following script has a Closure which should only be invoked once, at configuration time

task syncThirdPartyFolders(type: Sync) {
   configurations.thirdparty.files.each {
       println("File: " + it)
       from zipTree(it)
   }
   into "ThirdParty"
}
1 Like
3

To add to @Lance’s solution, you can avoid resolving the thirdparty configuration during configuration time by making a so-called “configuration task”.

task configSyncTPF {
    doLast {
        tasks.syncTPF.configure {
            configurations.thirdparty.each {
                println("File: " + it)
                from zipTree( it )
            }
            into 'ThirdParty'
        }
    }
}
task syncTPF( type: Sync, dependsOn: configSyncTPF )
1 Like