Need a little help with packaging individual dependencies in the build jar.
What I have:
A custom plugin called Base Plugin that applies Spring and other base dependencies.
A project called Core that just applies the earlier Base Plugin and nothing else. So automatically all the dependencies are added via the custom plugin. This project has a few utilities (e.g. validations etc.)
A project called App that applies the Base Plugin. This project also adds the earlier Core project as an implementation dependency.
When I do a gradle build on App, I get a slim jar without dependencies - which is expected.
All the dependencies mentioned in the custom plugin are already available to the app at runtime, so the slim jar created works as expected (without the need for a fat jar).
What I want:
I want ONLY the Core project added in the App projects’ jar file. When I do a gradle build, I should get App as well as Core in the jar file.
What I tried:
I followed earlier threads at How to include dependencies in jar? (How to include dependencies in jar?) and How to Fat Jar a Single Dependency (How to Fat Jar a Single Dependency). When I tried the solutions mentioned there, ALL the dependencies are added (including the ones in the custom plugin).
So, plugin jar is around 30 MB, Core is about 400 KB and App is about 200 KB. When I build it trying to add core, I am getting the entire 31 MB and not just the 600 KB I am looking for.
Is there any way to include ONLY the dependencies mentioned in the gradle build file and NOT the ones present in the plugin ? Or bundle only first level dependencies and DO NOT include transitive dependencies ?
PFB my App build.gradle
plugins {
id 'com.shrinathk.plugins.baseplugin' version '1.0'
}
group 'com.shrinathk'
version '1.0'
configurations {
extraLibs
}
dependencies {
extraLibs 'com.shrinathk:core:1.0'
configurations.compile.extendsFrom(configurations.extraLibs)
}
jar {
from {
configurations.compileClasspath.filter{
it.exists() }.collect {
it.isDirectory() ? it : zipTree(it) }
}
}
plugins {
id 'com.shrinathk.plugins.baseplugin' version '1.0'
}
group 'com.shrinathk'
version '1.0'
dependencies {
implementation 'com.shrinathk:core:1.0'
}
jar {
from {
configurations
.runtimeClasspath
.collect {
if(it.name.equalsIgnoreCase("core.jar")) {
zipTree(it)
}
}
}
}
If you use zipTree(it), the jar is expanded and the class files are merged in the jar.
If you just use it without zipTree, then the entire core.jar is copied as a jar itself into the app.jar.
If you want to see what all jars are merged into you final jar, you can use the following to print them and check: .collect { it.isDirectory() ? println ('Folder ' + it.name) : println ('File ' + it.name)}
The above solution has a problem. It merges the jars correctly, but while merging, the service files in META-INF/services are overwritten. Ideally they should be merged / concatenated. I tried using duplicatesStrategy(DuplicatesStrategy.INCLUDE) in the jar task, but it did not help.
In my case, the micronaut framework was defining the beans in a file called io.micronaut.inject.BeanDefinitionReference. When I merged these two projects, the app’s beans defined in the BeanDefinitionReference file were overwritten by the BeanDefinitionReference of the core. So I got a Bean Instantiation error at run time.
The only solution I have at this point is to use the com.github.johnrengelman.shadow plugin and use the mergeServiceFiles() option in it to merge the service files and manually exclude all other jars (using exclude 'io/**'exclude 'org/**' and so on) until I am left with only core and app jars.
Like so:
plugins {
id 'com.shrinathk.plugins.baseplugin' version '1.0'
id 'com.github.johnrengelman.shadow' version '6.1.0'
}
group 'com.shrinathk'
version '1.0'
dependencies {
implementation 'com.shrinathk:core:1.0'
}
shadowJar {
mergeServiceFiles()
exclude 'org/**'
exclude 'com/fasterxml/**'
}
I understand. But my requirement is fairly simple. My app depends on external jars and internal jars. External jars are supplied at runtime and hence need not be bundled. But internal jars (like our core or framework) cannot be supplied at runtime (for various reasons)… so I have no option but to bundle them.
I am definitely avoiding the fat jar (by not bundling external jars)… but for my use case, I do not see any other way than to bundle the app jar and the internal jars. If there is a better way, please do share. Thank you!
That depends too much on the exact specific situation to generically answer.
If you for example need one jar to put it into some system that requires that you deliver one jar, then you don’t have many other choices as the bug is the system requiring you to provide one jar.
If you for example are building some executable project, the application plugin would be the way to go.
…
Thanks for you inputs. I checked the application plugin. I am not planning to build an executable project, so that does not fit my use case.
I am going with slim jar only… the only reason I went for shadowPlugin and created a fatJar was because of the service files. Even in this case, the fat jar ONLY CONTAINS internal jars - so it’s still just over 150 KB.
@Vampire - Before using shadowJar, I did try to manually merge the service files. I was able to get them into one folder, but I was not able to merge / concatenate them inside the jar. PFB the snippet for the same.
plugins {
id 'com.shrinathk.plugins.baseplugin' version '1.0'
}
group 'com.shrinathk'
version '1.0'
dependencies {
implementation 'com.shrinathk:core:1.0'
}
jar {
from {
configurations
.runtimeClasspath
.collect {
if (it.name.equalsIgnoreCase("core.jar")) {
zipTree(it)
.each {
if (it.name.equalsIgnoreCase("io.micronaut.inject.BeanDefinitionReference")) {
// Creating a new file in the tmp directory
new File("build/tmp/io.micronaut.inject.core-project.BeanDefinitionReference")
.setText(it.getText())
}
}
}
}
}
// Copying the newly created file into the META-INF/services folder
from ('build/tmp/io.micronaut.inject.core-project.BeanDefinitionReference') {
into 'META-INF/services/'
}
}
The output of the snippet would be TWO BeanDefinitionRefernce files in the META-INF/services folder. But I was not able to merge / concatenate them.
If you can help me with a gradle task that can concatenate two files in-place inside the jar, I will call this task in the doLast method of the jar plugin and I can do away with the shadow plugin. Thank you!
That’s still a fat jar you building so there is not much won.
If you really need a fat jar, I would also use the shadow plugin.
I’m just a big disliker of fat jars in general.
I understand. But my requirement is fairly simple. My app depends on external jars and internal jars. External jars are supplied at runtime and hence need not be bundled. But internal jars (like our core or framework) cannot be supplied at runtime (for various reasons)… so I have no option but to bundle them.