Custom jar task producing archive with every *.class twice

I am sorry if this belongs to “Questions”, for me it appears as a problem but Im pretty sure it is caused by me doing something wrong :slight_smile: Im sorry that this will be longer text, I will try to be as brief as possible.

Here is the problem: I recently switched to Gradle because our project now has very specific build structure and Gradle seemed like excelent choice. In last three days I learned as much Groovy and Gradle as I can and solved most problems but Im stuck with last one - my custom jar tasks are producing jar files in which every *.class file is packaked twice (Im attaching screenshot of jar content).

Before I post the build script (as simplified as possible) I have to explain why it is done such way.

Our build structure and requirements:

  • one src directory with *.java sources

  • these sources represents one core and many components

  • what we need during build is to compile it alltogether but produce one jar for core and them component jars - one for every component

  • in jar we want to include sources also

  • there are many components and the number is raising -> build script should have component configuration at one place, the rest has to be done dynamically (upload stage is not important for this case)

I managed to create script that is working exactly as described but jar files contains every *.class twice, *.java files are ok (not duplicated). Im sure Im doing something wrong. Here is how my script works: - create own configuration ‘geCoreComponents’ - define big sourceSet containing all sources for compiling in one step and then one sourceSet for every component - dynamically create custom jar task for every component sourceSet. Jar task is configured with info from sourceSet’s config and is assigned to my ‘geCoreComponents’ configuration - assign my configuration to default jar task. I suppose here could be the problem. I suppose it would be better to create my own jar task with my configuration only as default contains also compile/runtime/test configuration. But I dont know how to do it

here is the script (im runinng it by ‘gradle compileGeCoreAllJava jar uploadGeCoreComponents’) apply plugin: ‘java’

project.version = “core” project.sourceCompatibility = 1.6 ext {

svnRevision = null

srcDir = “src”

outputDir = “bin”

svnAntLibDir = “lib”

compileCPLibsDir = “./…/GE_SharedLib_v7”

geCoreDistDir = “./…/GE_SharedLib_v7/ge_core”

}

sourceSets {

//main set which is used for compiling all source alltogether

geCoreAll {

java.srcDir file("$srcDir"); resources.srcDir file(“resources”); output.classesDir

= “$outputDir”; output.resourcesDir = “$outputDir”;

}

//component sourceset definitions. Used for creating component jar tasks which create jar files

core {

java.srcDirs = ["$srcDir/cz/trask/ge/digiflow/common", “$srcDir/cz/trask/ge/wps/”]

}

avizaDaily { java.srcDirs = ["$srcDir/cz/trask/ge/avizaDaily"] }

avizaWeekly { java.srcDirs = ["$srcDir/cz/trask/ge/avizaWeekly"] }

bankAtWork { java.srcDirs = ["$srcDir/cz/trask/ge/bankAtWork"] }

caprocessing { java.srcDirs = ["$srcDir/cz/trask/ge/caprocessing"] }

/* any more in original build script…*/ }

configurations {

geCoreComponents }

dependencies {

geCoreAllCompile fileTree(dir: compileCPLibsDir, include: ‘*.jar’) }

repositories {

flatDir {

name “geCoreLibsRepo”

dirs “$geCoreDistDir”

} }

compileGeCoreAllJava.doFirst {

tasks.withType(Compile) {

options.encoding = ‘UTF-8’

} }

/*

One jar task is created dynamically for every component sourceset and is attached to our ‘geCoreComponents’ configuration */ task createComponentJarTasks {

println “Creating component JAR tasks”

sourceSets.matching { it.name != “main” && it.name != “test” && it.name != “geCoreAll”}.each { set ->

String jarTaskName = “createJar_$set.name”

def jarTask = task("$jarTaskName", type: Jar) {

from sourceSets.geCoreAll.java.srcDirs //packaging sources

from sourceSets.geCoreAll.output //compiled classes

//every srcDir in defined in sourceSet is added to component jar task

set.java.srcDirs.each { dir ->

String componentClassesDir = (dir.path - projectDir.path).substring("$srcDir".length() + 2).replace(’\’, ‘/’) //get rid of ‘src/’

include("$componentClassesDir/**/*.class")

include("$componentClassesDir/**/*.java")

}

archiveName = “${set.name}.jar”

baseName = “$set.name”

manifest {

attributes(

“Implementation-Title”: “${set.name.toUpperCase()} GE_Core component”,

“built-by”: System.properties[‘user.name’])

}

}

artifacts {

geCoreComponents jarTask

}

} }

/* adding our configuration to jar task */ jar {

dependsOn configurations.geCoreComponents }

/* Custom upload task */ task uploadgeCoreComponents(type: Upload) {

setConfiguration(configurations.geCoreComponents)

uploadDescriptor = false

repositories {

flatDir {

name “geCoreLibsRepo”

dirs “$geCoreDistDir”

}

}

}

uploadgeCoreComponents.doFirst {

println “Uploading created JARs to $geCoreDistDir” }

String getRelativePath(String absolutePath) {

return (absolutePath - projectDir.path).substring(1).replace(’\’, ‘/’) }

If you get duplicate files in the archive, it means that you are pulling in the same files twice (via ‘from’). Currently, you have to guard against this yourself.

It has also crossed my mind but I dont see how - because I have only one from clause for every task: ‘from sourceSets.geCoreAll.output’. There is second ‘from’ clause but that is for *.java sources and if I completely disable it in the script it doesnt solve the problem - only effect is that *.java sources are not included.

And I tried to print the ‘source’ FileCollection attribute of every jar task I created and the colletion looks fine - every file is there only once so I have no idea how I manage to pull the duplicate into jar.

Hard for me to say. Maybe you have configured the same directory for sources and resources. Then I can see this happen.

Hopefully, we’ll soon have a better way to deal with duplicates. For now, you’ll have to hunt them down.

Resources shouldnt be the cause, but thanks for the tips, hopefully I will hunt it down and I will update this thread with my findings - in the case it will not be something very stupid at my side :slight_smile:

In the end it was caused by resources - I had same output directory for classes and resources for main sourceSet (‘output.classesDir = “$outputDir”; output.resourcesDir = “$outputDir”;’).

Maybee Im wrong but I guess that somehow gradle did second copy of each file as it treated all *.class files as resources and copied them all.

Maybee I dont know enough about jar tasks, but I dont the understand how it exactly happens, I dont see the mechanism behind this.

I thought that: 1. compileJava - sources are compiled and stored in ‘srcSet.output.classesDir’ 2. processResources - resources are copied to ‘srcSet.output.resourcesDir’ 3. the classesDir and resourcesDir is the same dir -> now all .class and .properties files are together in one directory 4. jar task with 'from srcSet.output; include("**/.class"); include("**/.properties")’ will copy all those files in one step and create target jar

But it seems like step 4 is executed twice - once for classes and once for resources and therefore all files are copied twice. Am I wrong ?

Thanks, Pavel

One Jar task can have multiple copy operations (i.e. 'from’s). There is a separate one for classes and resources.

Multiple from - I know but my created custom jar task has only one ‘from’ so I thought there will be only one copy operation.

One for classes and one for resources - so does this apply for all tasks with ‘(type: Jar)’ ? Sorry for these maybe stupid questions but if thats the case maybee Im blind but I didnt find that in docs so Im little confused about Jar task functionality.

It just applies to the ‘Jar’ task added by the Java plugin. However, I could imagine that ‘from srcSet.output’ results in two copy operations as well, as “output = classes + resources” (which aren’t necessarily sharing a common parent folder).

Ok thanks for patience and answers, I will try to remember this for the future :wink:

Two year later, any non guard yourself solutions? :slight_smile: (built-in duplicate resolution mechanism?)