Permission denied error during a copy in the processTestResource task on Linux server

Old Forum Archive Bugs
1

We started to upgrade from 2.4 to 2.12 and we started to see an issue with copying resources during the processTestResources task. I determined the first Gradle version to have this issue was version 2.5.
When we are doing a build on our Linux build server, we are getting a Permission denied when it is copying an xml from the source directory to the build directory. When I do a clean I can tell the file does not exist before the processTestReources task, but exists after it fails. But even though the file exists I get a failed permission error during the copy. This is not an issue on my local developer box which is Windows.

  • Exception is:
    org.gradle.api.tasks.TaskExecutionException: Execution failed for task ‘:paymentAllocation:processTestResources’.
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:203)
    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:185)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:66)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:50)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:25)
    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:110)
    at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
    at org.gradle.execution.DefaultBuildExecuter.access$000(DefaultBuildExecuter.java:23)
    at org.gradle.execution.DefaultBuildExecuter$1.proceed(DefaultBuildExecuter.java:43)
    at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:30)
    at org.gradle.initialization.DefaultGradleLauncher$4.run(DefaultGradleLauncher.java:154)
    at org.gradle.internal.Factories$1.create(Factories.java:22)
    at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
    at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:52)
    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:151)
    at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:32)
    at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:99)
    at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:93)
    at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
    at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:62)
    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:93)
    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:82)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:94)
    at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
    at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:43)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:28)
    at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:75)
    at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:45)
    at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:51)
    at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:28)

BUILD FAILED

Total time: 18.539 secs
at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:43)
at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:170)
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
at org.gradle.launcher.Main.doAction(Main.java:33)
at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:129)
at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)
Caused by: org.gradle.api.GradleException: Could not copy file ‘/srv/ccviews/p4/payment_allocation/main/paymentAllocation/src/test/resources/BaseDAO.xml’ to ‘/srv/ccviews/p4/payment_allocation/main/paymentAllocation/build/resources/test/BaseDAO.xml’.
at org.gradle.api.internal.file.AbstractFileTreeElement.copyTo(AbstractFileTreeElement.java:79)
at org.gradle.api.internal.file.copy.DefaultFileCopyDetails.copyTo(DefaultFileCopyDetails.java:104)
at org.gradle.api.internal.file.copy.FileCopyAction$FileCopyDetailsInternalAction.processFile(FileCopyAction.java:44)
at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator$1$1.processFile(NormalizingCopyActionDecorator.java:65)
at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator$1$1.processFile(DuplicateHandlingCopyActionDecorator.java:60)
at org.gradle.api.internal.file.copy.CopyFileVisitorImpl.processFile(CopyFileVisitorImpl.java:60)
at org.gradle.api.internal.file.copy.CopyFileVisitorImpl.visitFile(CopyFileVisitorImpl.java:44)
at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:81)
at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker$1.visitFile(Jdk7DirectoryWalker.java:54)
at java.nio.file.Files.walkFileTree(Files.java:2670)
at org.gradle.api.internal.file.collections.jdk7.Jdk7DirectoryWalker.walkDir(Jdk7DirectoryWalker.java:54)
at org.gradle.api.internal.file.collections.DirectoryFileTree.walkDir(DirectoryFileTree.java:146)
at org.gradle.api.internal.file.collections.DirectoryFileTree.visitFrom(DirectoryFileTree.java:130)
at org.gradle.api.internal.file.collections.DirectoryFileTree.visit(DirectoryFileTree.java:115)
at org.gradle.api.internal.file.collections.FileTreeAdapter.visit(FileTreeAdapter.java:109)
at org.gradle.api.internal.file.CompositeFileTree.visit(CompositeFileTree.java:58)
at org.gradle.api.internal.file.copy.CopySpecActionImpl.execute(CopySpecActionImpl.java:37)
at org.gradle.api.internal.file.copy.CopySpecActionImpl.execute(CopySpecActionImpl.java:24)
at org.gradle.api.internal.file.copy.DefaultCopySpec$DefaultCopySpecResolver.walk(DefaultCopySpec.java:498)
at org.gradle.api.internal.file.copy.DefaultCopySpec$DefaultCopySpecResolver.walk(DefaultCopySpec.java:500)
at org.gradle.api.internal.file.copy.DefaultCopySpec.walk(DefaultCopySpec.java:322)
at org.gradle.api.internal.file.copy.DelegatingCopySpecInternal.walk(DelegatingCopySpecInternal.java:206)
at org.gradle.api.internal.file.copy.CopySpecBackedCopyActionProcessingStream.process(CopySpecBackedCopyActionProcessingStream.java:36)
at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator$1.process(DuplicateHandlingCopyActionDecorator.java:44)
at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator$1.process(NormalizingCopyActionDecorator.java:56)
at org.gradle.api.internal.file.copy.FileCopyAction.execute(FileCopyAction.java:35)
at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator.execute(NormalizingCopyActionDecorator.java:52)
at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator.execute(DuplicateHandlingCopyActionDecorator.java:42)
at org.gradle.api.internal.file.copy.CopyActionExecuter.execute(CopyActionExecuter.java:38)
at org.gradle.api.tasks.AbstractCopyTask.copy(AbstractCopyTask.java:83)
at org.gradle.language.jvm.tasks.ProcessResources.copy(ProcessResources.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:227)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:220)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:209)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
… 68 more
Caused by: java.io.FileNotFoundException: /srv/ccviews/p4/payment_allocation/main/paymentAllocation/build/resources/test/BaseDAO.xml (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:213)
at java.io.FileOutputStream.(FileOutputStream.java:162)
at org.gradle.api.internal.file.AbstractFileTreeElement.copyFile(AbstractFileTreeElement.java:91)
at org.gradle.api.internal.file.AbstractFileTreeElement.copyTo(AbstractFileTreeElement.java:74)

2

I noticed there is a DuplicateStrategy class and I set it to FAIL for the processTestResources task. It failed during this task because it thought there was a duplicate. It seems to have each file more than once in the list of files to copy. I don’t understand why.

3

Turns out we had a plugin that that set the sourceSets.test.resources and they thought it would override the default setup. Turns out it appended the same directory again. This caused the copy functionality to copy multiple times since the same path was in the set.

I would say going forward I don’t see a use case to allow duplicate paths in the sourceSet. By definition of a Set I wouldn’t expect duplication.

4

We are still seeing this issue. If we don’t do a ‘clean’ command, Gradle does a copy of resource type files and because they already exist in the destination folder, Linux gives an error and fails the build. If I do a clean it works because the destination files no longer exist. But this defeats the idea of quick builds.
I am not sure what I can do about it.
Does anybody have any thoughts on this issue?
What are other people doing for builds on the Linux platform?

5

Hi Chris,

Could you provide more details about your build? Which Gradle version are you using by now? Is the cause of the problem still the plugin you mentioned? Can you tell us which plugin that is?

Maybe you can provide a build scan and/or a minimal example to reproduce the issue?

6

Thanks for replying.

Build version of Gradle: Something changed from 2.4 to 2.5 and it continues to exist all the way to current version.

Build steps to cause problem:

  1. Build on Linux (vs windows)
  2. Download code from source code repository (We are using Perforce)
  3. The files will be read only because it just came from Perforce. (very important detail)
  4. During the Test task, it copies reference files to a build directory. Gradle copy keeps the files with read only
    permissions. If this is the first time, then no issue. All subsequent times then Linux gives a permission error because you are not allowed to copy a file onto a read only file.

We just found a work around end of last week.
If we use the ALLWRITE option for Perforce then it sets the permissions of the file to write.
This allows Gradle to copy over existing files.
I am not sure if this should be the actual way a build should work.

Thoughts?

Thanks,
Chris

1 Like
7

Hi Chris,

Happy to hear that you found a solution that works for you.

I am also not able to reproduce this issue locally on OS X. So I would need more information to determine if (and what) the bug is here - and if it is still present in Gradle 3.1.

Maybe you can answer the following?

  • To confirm: you still experience the issue with Gradle 3.1?
  • To confirm: we are talking about files with permission 0440 (-r--r--r--) in src/test/resources/ ?
  • On which Linux version and file system do you encounter the problem?

Since it seems to work in general (on your and also on my environment), is it maybe an issue with the user that runs the build on your linux environment (something like missing the rights to change file permissions)?