I’m trying to copy a 1MB text file from a network share. Sometimes it takes 1 second, sometimes 20 and sometimes up to 40 seconds. If I copy the same file with Windows Explorer / Ant / batch file it works fine.
This is the code:
def source = '//myHost/test/'
def dest = 'c:/test/'
task copyTask() << {
copy {
from source
into dest
}
}
In general, it’s more appropriate to use a ‘Copy’ task, but I doubt it will have a positive effect on performance. Have you tried this often enough to be absolutely sure that the performance problem is related to Gradle? I’d be surprised if it was. Of course, Windows Explorer and batch files may have an advantage because they can use the Windows APIs directly, instead of going through the Java APIs. Do you see anything interesting in the Gradle ‘–info’ or ‘–debug’ logs?
Below are 3 runs with --info We can see that they took 18 seconds, then 2 seconds, then 20 seconds. The file size is a little less than 5MB
Code:
def source = '//uriaa01v/Public/test/'
def dest = 'c:/test/'
task remoteCopy(type: Copy) {
delete dest
from source
into dest
}
Output:
C:\BSS\v90_0\localEnv\remoteCopy>gradle remoteCopy --info
Starting Build
Settings evaluated using empty settings script.
Projects loaded. Root project using build file 'C:\BSS\v90_0\localEnv\remoteCopy\build.gradle'.
Included projects: [root project 'remoteCopy']
Evaluating root project 'remoteCopy' using build file 'C:\BSS\v90_0\localEnv\remoteCopy\build.gradle'.
All projects evaluated.
Selected primary task 'remoteCopy'
Tasks to be executed: [task ':remoteCopy']
:remoteCopy
Executing task ':remoteCopy' due to:
Output file C:\test for task ':remoteCopy' has changed.
Output file C:\test\test3a.txt has been removed for task ':remoteCopy'.
BUILD SUCCESSFUL
Total time: 18.136 secs
C:\BSS\v90_0\localEnv\remoteCopy>gradle remoteCopy --info
Starting Build
Settings evaluated using empty settings script.
Projects loaded. Root project using build file 'C:\BSS\v90_0\localEnv\remoteCopy\build.gradle'.
Included projects: [root project 'remoteCopy']
Evaluating root project 'remoteCopy' using build file 'C:\BSS\v90_0\localEnv\remoteCopy\build.gradle'.
All projects evaluated.
Selected primary task 'remoteCopy'
Tasks to be executed: [task ':remoteCopy']
:remoteCopy
Executing task ':remoteCopy' due to:
Output file C:\test for task ':remoteCopy' has changed.
Output file C:\test\test3a.txt has been removed for task ':remoteCopy'.
BUILD SUCCESSFUL
Total time: 2.218 secs
C:\BSS\v90_0\localEnv\remoteCopy>gradle remoteCopy --info
Starting Build
Settings evaluated using empty settings script.
Projects loaded. Root project using build file 'C:\BSS\v90_0\localEnv\remoteCopy\build.gradle'.
Included projects: [root project 'remoteCopy']
Evaluating root project 'remoteCopy' using build file 'C:\BSS\v90_0\localEnv\remoteCopy\build.gradle'.
All projects evaluated.
Selected primary task 'remoteCopy'
Tasks to be executed: [task ':remoteCopy']
:remoteCopy
Executing task ':remoteCopy' due to:
Output file C:\test for task ':remoteCopy' has changed.
Output file C:\test\test3a.txt has been removed for task ':remoteCopy'.
BUILD SUCCESSFUL
Total time: 20.667 secs
C:\BSS\v90_0\localEnv\remoteCopy>
and below is output with --debug which took 15 seconds:
C:\BSS\v90_0\localEnv\remoteCopy>gradle remoteCopy --debug
10:35:47.969 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:35:48.594 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:35:48.594 [INFO] [org.gradle.BuildLogger] Starting Build
10:35:48.609 [DEBUG] [org.gradle.BuildLogger] Gradle user home: C:\Documents and Settings\uriaa\.gradle
10:35:48.609 [DEBUG] [org.gradle.BuildLogger] Current dir: C:\BSS\v90_0\localEnv\remoteCopy
10:35:48.609 [DEBUG] [org.gradle.BuildLogger] Settings file: null
10:35:48.609 [DEBUG] [org.gradle.BuildLogger] Build file: null
10:35:48.625 [DEBUG] [org.gradle.initialization.BuildSourceBuilder] Starting to build the build sources.
10:35:48.640 [DEBUG] [org.gradle.initialization.BuildSourceBuilder] Gradle source dir does not exist. We leave.
10:35:48.656 [DEBUG] [org.gradle.initialization.DefaultGradlePropertiesLoader] Found env project properties: []
10:35:48.672 [DEBUG] [org.gradle.initialization.DefaultGradlePropertiesLoader] Found system project properties: []
10:35:48.797 [DEBUG] [org.gradle.api.internal.artifacts.mvnsettings.DefaultLocalMavenRepositoryLocator] No local repository in Settings file defined. Using defa
ult path: C:\Documents and Settings\uriaa\.m2\repository
10:35:49.062 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:35:49.078 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:35:49.078 [DEBUG] [org.gradle.initialization.ScriptEvaluatingSettingsProcessor] Timing: Processing settings took: 0.406 secs
10:35:49.093 [INFO] [org.gradle.BuildLogger] Settings evaluated using empty settings script.
10:35:49.172 [DEBUG] [org.gradle.initialization.ProjectPropertySettingBuildLoader] Looking for project properties from: C:\BSS\v90_0\localEnv\remoteCopy\gradle.
properties
10:35:49.172 [DEBUG] [org.gradle.initialization.ProjectPropertySettingBuildLoader] project property file does not exists. We continue!
10:35:49.172 [INFO] [org.gradle.BuildLogger] Projects loaded. Root project using build file 'C:\BSS\v90_0\localEnv\remoteCopy\build.gradle'.
10:35:49.187 [INFO] [org.gradle.BuildLogger] Included projects: [root project 'remoteCopy']
10:35:49.203 [INFO] [org.gradle.configuration.BuildScriptProcessor] Evaluating root project 'remoteCopy' using build file 'C:\BSS\v90_0\localEnv\remoteCopy\buil
d.gradle'.
10:35:49.203 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on buildscript class cache for build file 'C:\BSS\v90_0\l
ocalEnv\remoteCopy\build.gradle' (C:\Documents and Settings\uriaa\.gradle\caches.4\scripts\build_43kbbr74ti3k733hje545922l4\ProjectScript\buildscript).
10:35:49.218 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired.
10:35:49.234 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:35:49.250 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on no_buildscript class cache for build file 'C:\BSS\v90_
0\localEnv\remoteCopy\build.gradle' (C:\Documents and Settings\uriaa\.gradle\caches.4\scripts\build_43kbbr74ti3k733hje545922l4\ProjectScript\no_buildscript).
10:35:49.250 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired.
10:35:49.375 [DEBUG] [org.gradle.api.internal.file.copy.DeleteActionImpl] Deleting C:\test
10:35:49.390 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:35:49.390 [DEBUG] [org.gradle.configuration.BuildScriptProcessor] Timing: Running the build script took 0.187 secs
10:35:49.578 [INFO] [org.gradle.BuildLogger] All projects evaluated.
10:35:49.593 [DEBUG] [org.gradle.execution.taskgraph.DefaultTaskGraphExecuter] Timing: Creating the DAG took 0.0 secs
10:35:49.609 [INFO] [org.gradle.execution.TaskNameResolvingBuildConfigurationAction] Selected primary task 'remoteCopy'
10:35:49.625 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on task artifact state cache (C:\BSS\v90_0\localEnv\re
moteCopy\.gradle.4\taskArtifacts).
10:35:49.625 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired.
10:35:49.625 [INFO] [org.gradle.BuildLogger] Tasks to be executed: [task ':remoteCopy']
10:35:49.640 [LIFECYCLE] [org.gradle.TaskExecutionLogger] :remoteCopy
10:35:49.640 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Starting to execute task ':remoteCopy'
10:35:49.734 [DEBUG] [org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter] Determining if task ':remoteCopy' is up-to-date
10:35:49.750 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache taskArtifacts.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4
\taskArtifacts\taskArtifacts.bin)
10:35:49.781 [DEBUG] [org.gradle.api.internal.changedetection.DefaultFileCacheListener] Can cache files for directory '\uriaa01v\Public\test'
10:35:49.796 [DEBUG] [org.gradle.api.internal.changedetection.DefaultFileCacheListener] Can cache files for file 'C:\test'
10:35:49.796 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache outputFileStates.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle\
1.4\taskArtifacts\outputFileStates.bin)
10:35:49.843 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache fileHashes.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4\ta
skArtifacts\fileHashes.bin)
10:35:49.859 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache fileSnapshots.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4
\taskArtifacts\fileSnapshots.bin)
10:35:49.875 [INFO] [org.gradle.api.internal.changedetection.DefaultTaskArtifactStateRepository] Executing task ':remoteCopy' due to:
Output file C:\test for task ':remoteCopy' has changed.
Output file C:\test\test3a.txt has been removed for task ':remoteCopy'.
10:35:49.875 [DEBUG] [org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter] task ':remoteCopy' is not up-to-date
10:35:49.890 [DEBUG] [org.gradle.api.internal.changedetection.DefaultFileCacheListener] Invalidate cached files for file 'C:\test'
10:35:49.890 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache fileSnapshots.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4
\taskArtifacts\fileSnapshots.bin)
10:35:49.890 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache taskArtifacts.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4
\taskArtifacts\taskArtifacts.bin)
10:35:49.906 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache fileHashes.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4\ta
skArtifacts\fileHashes.bin)
10:35:49.906 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache outputFileStates.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle\
1.4\taskArtifacts\outputFileStates.bin)
10:35:49.906 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on task artifact state cache (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.
4\taskArtifacts).
10:35:49.921 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter] Executing actions for task ':remoteCopy'.
10:35:49.921 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:35:49.937 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:36:02.809 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
10:36:02.824 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on task artifact state cache (C:\BSS\v90_0\localEnv\re
moteCopy\.gradle.4\taskArtifacts).
10:36:02.824 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired.
10:36:02.840 [DEBUG] [org.gradle.api.internal.changedetection.DefaultFileCacheListener] Can cache files for file 'C:\test'
10:36:02.840 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache outputFileStates.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle\
1.4\taskArtifacts\outputFileStates.bin)
10:36:02.856 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache fileHashes.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4\ta
skArtifacts\fileHashes.bin)
10:36:02.887 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache fileSnapshots.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4
\taskArtifacts\fileSnapshots.bin)
10:36:02.902 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache taskArtifacts.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4
\taskArtifacts\taskArtifacts.bin)
10:36:02.902 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':remoteCopy'
10:36:02.918 [DEBUG] [org.gradle.execution.taskgraph.DefaultTaskGraphExecuter] Timing: Executing the DAG took 13.293 secs
10:36:02.918 [LIFECYCLE] [org.gradle.BuildResultLogger]
10:36:02.918 [LIFECYCLE] [org.gradle.BuildResultLogger] BUILD SUCCESSFUL
10:36:02.934 [LIFECYCLE] [org.gradle.BuildResultLogger]
10:36:02.934 [LIFECYCLE] [org.gradle.BuildResultLogger] Total time: 15.215 secs
10:36:02.934 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache fileSnapshots.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4
\taskArtifacts\fileSnapshots.bin)
10:36:02.949 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache taskArtifacts.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4
\taskArtifacts\taskArtifacts.bin)
10:36:02.949 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache fileHashes.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.4\ta
skArtifacts\fileHashes.bin)
10:36:02.949 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache outputFileStates.bin (C:\BSS\v90_0\localEnv\remoteCopy\.gradle\
1.4\taskArtifacts\outputFileStates.bin)
10:36:02.965 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on task artifact state cache (C:\BSS\v90_0\localEnv\remoteCopy\.gradle.
4\taskArtifacts).
10:36:02.965 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on no_buildscript class cache for build file 'C:\BSS\v90_0\localEnv\remot
eCopy\build.gradle' (C:\Documents and Settings\uriaa\.gradle\caches.4\scripts\build_43kbbr74ti3k733hje545922l4\ProjectScript\no_buildscript).
10:36:02.981 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on buildscript class cache for build file 'C:\BSS\v90_0\localEnv\remoteCo
py\build.gradle' (C:\Documents and Settings\uriaa\.gradle\caches.4\scripts\build_43kbbr74ti3k733hje545922l4\ProjectScript\buildscript).
10:36:02.996 [DEBUG] [org.gradle.logging.internal.DefaultLoggingConfigurer] Finished configuring with level: DEBUG, configurers: [org.gradle.logging.internal.Ou
tputEventRenderer@6da8eb, org.gradle.logging.internal.logback.LogbackLoggingConfigurer@16d079a, org.gradle.logging.internal.JavaUtilLoggingConfigurer@10e468f]
C:\BSS\v90_0\localEnv\remoteCopy>
Also, I tried it many times using Gradle and windows explorer. In windows explorer it consistently takes less than 3 seconds for that 5MB file In gradle, it is consistently inconsistent and can take 2 or 10 or 20 seconds
The real life task I need to accomplish is deploying tools and a large SDK to local development machines. The total size of these resources is more than 1.5GB so I must be able to copy these files as quickly as possible.
Unfortunately I can no longer reply to this topic because the forums can’t handle long posts (please use an external service like Gist when posting logs). Could you open another topic?