1.7-rc-1: Probably erroneous "Including duplicate file" warnings

huxi · July 4, 2013, 9:30am

I get quite a bunch of “Including duplicate file” warnings from the ‘processResources’ task even though I’m pretty sure that there are no duplicates.

The task configuration in question looks like this:

processResources {
  outputs.upToDateWhen { false } // Workaround for http://jira.codehaus.org/browse/GRADLE-1276
  from(sourceSets.main.resources.srcDirs) {
   exclude '**/*.properties'
   exclude '**/*.xhtml'
   }
    from(sourceSets.main.resources.srcDirs) {
   include '**/*.properties'
   include '**/*.xhtml'
   expand(
    'pom': [name: 'Lilith', version: version],
    'lilithVersion': version,
    'slf4jVersion': versions.slf4j,
    'logbackVersion': versions.logback,
    'gitRevision': project.gitHeadHash,
    'timestamp': System.currentTimeMillis()
   )
  }
 }

(See https://github.com/huxi/lilith/blob/master/build.gradle#L347 )

Warnings are emitted for every resource file, even the ones explicitly excluded/included above.

This is the same module that is affected by GRADLE-1298 , so ‘clean’ is always executed before building it - not sure if this is relevant.

luke_daley · July 4, 2013, 9:58am

Duplicates might not be respecting includes.

Would you be able to provide a small sample I can use?

huxi · July 4, 2013, 10:29am

Not immediately, probably some hours later today.

The interesting thing is that both files mentioned in the explicit include/exclude as well as “normal” files are mentioned in the warning. So the warning may be triggered by the mere existence of multiple ‘from’ entries.

Just guessing, of course. I’ll take a look into the issue this evening if you still need a sample by then.

luke_daley · July 4, 2013, 4:19pm

This isn’t a bug, you do have duplicates. The processResources task is already pulling from ‘sourceSets.main.resources.srcDirs’ and you’re adding it again.

You should change your task to:

processResources {
  def props = [
    'pom': [name: 'Lilith', version: version],
    'lilithVersion': version,
    'slf4jVersion': versions.slf4j,
    'logbackVersion': versions.logback,
    'gitRevision': project.gitHeadHash,
    'timestamp': System.currentTimeMillis()
  ]
  input.property 'props', props
  filesMatching '**/*.properties', { expand props }
  filesMatching '**/*.xhtml', { expand props }
}

luke_daley · July 4, 2013, 4:28pm

btw, this task will always be out of date due to the timestamp.

huxi · July 6, 2013, 2:39pm

That code doesn’t work. I tried to change ‘input.property’ to ‘inputs.property’ but this results in the following exception:

org.gradle.api.UncheckedIOException: Could not add entry ':lilith:processResources' to cache taskArtifacts.bin (/Users/huxi/Documents/Projects/huxi/lilith/.gradle/1.7-rc-1/taskArtifacts/taskArtifacts.bin).
 at org.gradle.cache.internal.btree.BTreePersistentIndexedCache.put(BTreePersistentIndexedCache.java:152)
 at org.gradle.cache.internal.MultiProcessSafePersistentIndexedCache$2.run(MultiProcessSafePersistentIndexedCache.java:53)
 at org.gradle.cache.internal.DefaultFileLockManager$DefaultFileLock.doWriteAction(DefaultFileLockManager.java:199)
 at org.gradle.cache.internal.DefaultFileLockManager$DefaultFileLock.writeFile(DefaultFileLockManager.java:188)
 at org.gradle.cache.internal.DefaultCacheAccess$UnitOfWorkFileAccess.writeFile(DefaultCacheAccess.java:365)
 at org.gradle.cache.internal.MultiProcessSafePersistentIndexedCache.put(MultiProcessSafePersistentIndexedCache.java:51)
 at sun.reflect.GeneratedMethodAccessor473.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at org.gradle.listener.LazyCreationProxy$LazyInvocationHandler.invoke(LazyCreationProxy.java:49)
 at com.sun.proxy.$Proxy23.put(Unknown Source)
 at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository$1$1.run(CacheBackedTaskHistoryRepository.java:79)
 at org.gradle.internal.Factories$1.create(Factories.java:22)
 at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:143)
 at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:131)
 at org.gradle.cache.internal.DefaultPersistentDirectoryStore.useCache(DefaultPersistentDirectoryStore.java:134)
 at org.gradle.api.internal.changedetection.state.DefaultTaskArtifactStateCacheAccess.useCache(DefaultTaskArtifactStateCacheAccess.java:74)
 at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository$1.update(CacheBackedTaskHistoryRepository.java:62)
 at org.gradle.api.internal.changedetection.changes.DefaultTaskArtifactStateRepository$TaskArtifactStateImpl.afterTask(DefaultTaskArtifactStateRepository.java:118)
 at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
 at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
 at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
 at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
 at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
 at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
 at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:286)
 at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:86)
 at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:72)
 at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTaskWithCacheLock(AbstractTaskPlanExecutor.java:64)
 at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:54)
 at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$1.run(DefaultTaskPlanExecutor.java:34)
 at org.gradle.internal.Factories$1.create(Factories.java:22)
 at org.gradle.cache.internal.DefaultCacheAccess.longRunningOperation(DefaultCacheAccess.java:214)
 at org.gradle.cache.internal.DefaultCacheAccess.longRunningOperation(DefaultCacheAccess.java:276)
 at org.gradle.cache.internal.DefaultPersistentDirectoryStore.longRunningOperation(DefaultPersistentDirectoryStore.java:142)
 at org.gradle.api.internal.changedetection.state.DefaultTaskArtifactStateCacheAccess.longRunningOperation(DefaultTaskArtifactStateCacheAccess.java:78)
 at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:32)
 at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:89)
 at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
 at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)
 at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
 at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:67)
 at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
 at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)
 at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:54)
 at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:166)
 at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:113)
 at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:81)
 at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:64)
 at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
 at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
 at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:35)
 at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:45)
 at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:42)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:24)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.StartStopIfBuildAndStop.execute(StartStopIfBuildAndStop.java:33)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.ReturnResult.execute(ReturnResult.java:34)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:70)
 at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:68)
 at org.gradle.util.Swapper.swap(Swapper.java:38)
 at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:68)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)
 at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:59)
 at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:45)
 at org.gradle.launcher.daemon.server.DaemonStateCoordinator.runCommand(DaemonStateCoordinator.java:186)
 at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy.doBuild(StartBuildOrRespondWithBusy.java:49)
 at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.HandleStop.execute(HandleStop.java:36)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.CatchAndForwardDaemonFailure.execute(CatchAndForwardDaemonFailure.java:32)
 at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:125)
 at org.gradle.launcher.daemon.server.exec.DefaultDaemonCommandExecuter.executeCommand(DefaultDaemonCommandExecuter.java:48)
 at org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler$ConnectionWorker.handleCommand(DefaultIncomingConnectionHandler.java:155)
 at org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler$ConnectionWorker.receiveAndHandleCommand(DefaultIncomingConnectionHandler.java:128)
 at org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler$ConnectionWorker.run(DefaultIncomingConnectionHandler.java:116)
 at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:66)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:724)
Caused by: java.io.NotSerializableException: java.io.PrintWriter
 at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1181)
 at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
 at java.util.HashMap.writeObject(HashMap.java:1100)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
 at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
 at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
 at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
 at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
 at java.util.HashMap.writeObject(HashMap.java:1100)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
 at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
 at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
 at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
 at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
 at org.gradle.messaging.serialize.DefaultSerializer.write(DefaultSerializer.java:51)
 at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository$LazyTaskExecution$Serializer.write(CacheBackedTaskHistoryRepository.java:266)
 at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository$TaskHistorySerializer.write(CacheBackedTaskHistoryRepository.java:156)
 at org.gradle.api.internal.changedetection.state.CacheBackedTaskHistoryRepository$TaskHistorySerializer.write(CacheBackedTaskHistoryRepository.java:134)
 at org.gradle.messaging.serialize.DataStreamBackedSerializer.write(DataStreamBackedSerializer.java:29)
 at org.gradle.cache.internal.btree.BTreePersistentIndexedCache$DataBlock.setValue(BTreePersistentIndexedCache.java:648)
 at org.gradle.cache.internal.btree.BTreePersistentIndexedCache$DataBlock.useNewValue(BTreePersistentIndexedCache.java:683)
 at org.gradle.cache.internal.btree.BTreePersistentIndexedCache.put(BTreePersistentIndexedCache.java:140)
 ... 90 more

That task is supposed to be out of date. It’s just not nice that the module has to recompile all the source.

huxi · July 7, 2013, 2:36pm

Found the solution:

processResources {
   def props = [
    'lilithVersion': version,
    'slf4jVersion': versions.slf4j,
    'logbackVersion': versions.logback,
    'gitRevision': project.gitHeadHash,
    'timestamp': "${System.currentTimeMillis()}"
   ]
  inputs.properties props
   filesMatching '**/*.properties', { expand props }
   filesMatching '**/*.xhtml', { expand props }
  }

‘input.property ‘props’, props’ needs to be replaced by ‘inputs.properties props’.

That’s a nice workaround and prevents that I have to recompile all the time. Thanks!

luke_daley · July 26, 2013, 9:41am

These warnings are gone in 1.7.

Topic		Replies	Views
How to I conditionally filter a file during the "processResources" stage (1.7 warns of a duplicate file) Old Forum Archive	2	1665	July 3, 2013
Unexpected "Duplicate path" warning in :processFuncTestResources task Help/Discuss	6	4583	August 18, 2022
Best practices on configure resources Help/Discuss	1	313	April 3, 2020
Gradle 6, Gradle TestKit and DuplicatesStrategy.INCLUDE warning Help/Discuss	0	611	February 29, 2020
Using sourceSets along with Jar task in gradle build, causing build failure in windows due to duplicateStrategy not being set Help/Discuss	7	939	November 20, 2023

1.7-rc-1: Probably erroneous "Including duplicate file" warnings

Related topics