Hello,
I would like to make specific directory layout for downloaded jars. It’s important as we want to be able to change fast and easily Java classpath on server. Proper directory structure of jars will make it easier task.
Good example is logging with slf4j. I would like to achieve following directory structure in lib:
+--- logging
|
+ -
logback
|
|
| - logback-classic-1.0.7.jar
|
|
\ - logback-core-1.0.7.jar
|
|
|
+ -
log4j
|
|
\ - slf4j-log4j12-1.7.0.jar
|
|
|
\ - slf4j-api-1.7.0.jar
|
| ....
Is there a good way to configure Gradle to make something like this?
One way which I can make it is to use 3 configurations for logging:
logging_in_use 'ch.qos.logback:logback-classic:1.0.7'
logging_in_use 'ch.qos.logback:logback-core:1.0.7'
logging_alter1 'org.slf4j:slf4j-log4j12:1.7.0'
logging_common 'org.slf4j:slf4j-api:1.7.0'
logging_common 'log4j:log4j:1.2.17'
and then mess with configurations: add logging_in_use and logging_common to classpath, only download alter1 [and other alternatives if necessary] and finally copy all files into distribution. But in fact it is very ugly way to go.
How to do it better?
/Marcin
No better solution? Has someone similar needs?
I don’t fully understand what you are trying to accomplish. What is it that’s shipped to the server, and how does the lib directory relate to this?
Thanks for asking, I will try to explain in more details, but if something is still unclear, please do not hesitate to ask.
We want to be able to use in our application two different api implementation for logging. In our case it might be implementation of slf4j: either slf4j-log4j or slf4j-logback. I will use this example but in fact we have also other cases e.g. using ibm mom version 6.5 or version 7.0. Or maybe in some other case e.g. different implementation of jdbc backend. What is important is that all artifacts (jars) have to be deployed on server, so that it is possible to change implementation quickly. In our case we have some scripts to make server restart and in this script we just change Java class path:
JAVA_CLASSPATH="./conf:./hpa-core.jar:./lib/*:./lib/oracle/*:./lib/mom/*:./lib/mom/7.5.0.0/*:./lib/logging/*:./lib/logging/log4j/*:./lib/shards/*"
Having all artifacts divided into subdirectories makes changing class path a snap comparing to cherry-picking proper jars from jar soup.
Currently slf4j jars are just committed into SVN. It is simple and works properly, but it is just Bad Thing ™.
In our case deploying application on server is also a matter of executing some scripts (perl, bash or something else…). These scripts expects to have zipped distribution of built project, and this is where Gradle is used.
I have defined createDistributionZip task which copy artifacts and other files necessary for deployment into one zip. This zip file is basically unpacked on server. … And I have a problem how to define dependencies in Gradle script, but then, when creating distribution, how to divide artifacts into proper directories. In fact similar feature would be useful also for working in Eclipse - possibility to change jar set in eclipse classpath depending on which e.g. implementation of slf4j I would like to use.
I have an idea how Gradle could be improved for such a use-cases, but first I would like to hear if there are other options.
My gut feeling is that this is best solved with some scripting, rather than a new Gradle feature. Something like:
ext.libraries = [
logging: [
common: ['org.slf4j:slf4j-api:1.7.0', 'log4j:log4j:1.2.17']
alternatives: [
logback: ['ch.qos.logback:logback-classic:1.0.7', 'ch.qos.logback:logback-core:1.0.7'],
log4j: ['org.slf4j:slf4j-log4j12:1.7.0']
]
]
]
Such a data structure could then be used to:
- Add common libraries and first alternatives to the ‘compile’ configuration. * Have a task create a lib directory structure. * Have a task generate the desired class path, e.g. by executing 'gradle assembleClasspath --alternatives logging.log4j,mom.ibm6_5 (potentially on the server). This could create a new lib directory, or just print out a class path based on an existing lib directory (which could potentially stay flat).
Sorry for not answering for a long time, but I wanted to test your tip before coming back to you.
I started implementing this approach, and it seems promising. My solution is as below:
apply from: 'common.gradle'
//---------------------------
//Common code
enum Entry {
COMMON, DEFAULT, ALTERNATIVE
}
def apply(Map libraries, closure) {
for ( group in libraries ) {
List common = group.value.getAt( 'common' )
for ( lib in common ) {
closure( Entry.COMMON, group.key, 'common', lib )
}
Map alternatives = group.value.getAt( 'alternatives' )
for ( alternative in alternatives ) {
Entry entry
String alternativeGroup
if (alternative.key.startsWith('!')) {
alternativeGroup = alternative.key.substring(1, alternative.key.size())
entry = Entry.DEFAULT
} else {
alternativeGroup = alternative.key
entry = Entry.ALTERNATIVE
}
for (lib in alternative.value) {
closure( entry, group.key, alternativeGroup, lib )
}
}
}
}
def printFn = { entry, group, name, lib ->
println("$entry - $group - $name - $lib")
}
def compileFn = { entry, group, name, lib ->
switch (entry) {
case Entry.COMMON:
case Entry.DEFAULT:
dependencies.compile lib
break
}
}
//---------------------------
//Specific code
Map altLibraries = [
'logging': [
'common': ['org.slf4j:slf4j-api:1.7.0', 'log4j:log4j:1.2.17'],
'alternatives': [
'!logback': ['ch.qos.logback:logback-classic:1.0.7', 'ch.qos.logback:logback-core:1.0.7'],
'log4j': ['org.slf4j:slf4j-log4j12:1.7.0']
]
]
]
dependencies {
compile 'org.apache.commons:commons-lang3:3.1'
compile 'commons-cli:commons-cli:1.2'
compile 'commons-io:commons-io:2.4'
compile 'com.google.guava:guava:13.0.1'
compile 'joda-time:joda-time:2.1'
compile 'com.thoughtworks.xstream:xstream:1.4.3'
apply(altLibraries, printFn)
apply(altLibraries, compileFn)
}
This code works for now perfectly good for me, but now I would like to move all common code into file common.gradle. The problem is that this code is not visible in build.gradle file. (Namely functions: apply, printFn and compileFn)
How can I in a best way make it visible in build.gradle?
Use a closure instead of a method (like you already do in ‘printFn’ and ‘compileFn’), and an extension property instead of a local variable (‘ext.compileFn = …’). Callers can then just use ‘compileFn(…)’.
I have changed my code as follows (I put here only relevant changes):
build.gradle:
ext.alternativeLibraries = [
'logging': [
'common': ['org.slf4j:slf4j-api:1.7.0', ......
]
dependencies {
....
ext.applyAlternativeDependencies()
....
}
-------------------------------------------------------------
common.gradle
ext.applyAlternativeDependencies = {
apply(ext.alternativeLibraries, compileDependenciesFn)
}
Now I am trying to change distribution task, which should copy libraries into structure:
common.gradle
task createDistributionDir(dependsOn: jar,type : Copy) {
String distDir = "$buildDir/distributions/$baseLine"
from(fileTree(".") {
include "conf/**"
include "lib/**"
})
from(fileTree("externals") {
include "lib/**"
exclude "lib/test/**"
exclude "lib/sources/**"
})
from("install")
from(jar.archivePath)
from(configurations.sabreLibs) { into "lib" }
from(configurations.mavenLibs) { into "lib" }
apply(ext.alternativeLibraries) { entry, group, name, lib ->
switch (entry) {
case Entry.COMMON:
case Entry.DEFAULT:
case Entry.ALTERNATIVE:
break
}
}
}
First problem is that I get an error: > cannot get property ‘alternativeLibraries’ on extra properties extension as it does not exist
Please notice that I can not move alternativeLibraries into common.gradle, as it’s proper place is in build.gradle - theoretically every project can have it’s own configuration for these libraries.
Another problem is that I don’t have a source path for my libraries. E.g. logging libraries should be downloaded from maven central (they are stored in Gradle cache), but Mom libraries should be copied from local store (in our case it is ‘flatDir’ in our SVN repository).
How to solve these issues?
‘ext.’ should only be used when writing extra properties, not when reading them. If the plugin script needs to access variables set in the main script, then you either need to apply it after setting those variables, or use one of the techniques for deferring evaluation (wrap code passed to methods like ‘CopySpec.from’ in a closure (’{…}’), wrap code in ‘project.afterEvaluate {…}’, etc.).
Unfortunately it doesn’t seem to be simple and easy to maintain… Can you please suggest how to solve my second issue? I still don’t have access to libraries, which are passed as string into closure:
task createDistributionDir(...... {
apply(ext.alternativeLibraries) { entry, group, name, lib ->
switch (entry) {
case Entry.COMMON: ????
case Entry.DEFAULT: ????
case Entry.ALTERNATIVE: ????
break
}
}
}
Let’s say I already have access to libraries definition (ext.alternativeLibraries) in above closure. What should I put in switch cases, to copy libraries to "lib/$group/$name/ ???
If you could provide code snippets it would be really helpful - Groovy is new for me, and I don’t have also so much experiences with Gradle.
PS. Initially I was thinking about adding property like ‘ext’ into configuration. It could be e.g. map with custom properties. Then it would be probably easier to define dependency and read it. It should be possible also to filter configuration by these properties. (I can not define details, because I don’t know enough neither Groovy, nor Gradle. So it is just a sketch):
dependencies {
compile 'org.slf4j:slf4j-api:1.7.0', path: '+lib/logging/'
compile 'log4j:log4j:1.2.17', path: '+lib/logging/'
compile 'ch.qos.logback:logback-classic:1.0.7', path: '+lib/logging/logback/'
compile 'ch.qos.logback:logback-core:1.0.7', path: '+lib/logging/logback/'
compile 'org.slf4j:slf4j-log4j12:1.7.0', path: '-lib/logging/log4j/'
}
(+/- indicates if it is default or not)
What do you think?
I have tried the following:
build.gradle
// Libraries with alternatives - (! default library)
ext.alternativeLibraries = [
'logging': [
'common': ['org.slf4j:slf4j-api:1.7.0', 'log4j:log4j:1.2.17'],
'alternatives': [
'logback': ['ch.qos.logback:logback-classic:1.0.7', 'ch.qos.logback:logback-core:1.0.7'],
'!log4j': ['org.slf4j:slf4j-log4j12:1.7.0']
]
]
]
apply from: 'externals/gradle/alternatives.gradle'
and it doesn’t work. I get error: > cannot get property ‘alternativeLibraries’ on extra properties extension as it does not exist
on:
alternatives.gradle
task createDistributionDir(dependsOn: jar,type : Copy) {
....
apply(ext.alternativeLibraries) { entry, group, name, lib ->
....
}
....
}
Ok. Finally it seems that my solution works:
build.gradle (fragments):
version = '1.0'
// Libraries with alternatives - (! default library)
ext.alternativeLibraries = [
'logging': [
'common': ['org.slf4j:slf4j-api:1.7.2', 'log4j:log4j:1.2.17'],
'alternatives': [
'logback': ['ch.qos.logback:logback-classic:1.0.9', 'ch.qos.logback:logback-core:1.0.9'],
'!log4j': ['org.slf4j:slf4j-log4j12:1.7.2']
]
]
]
apply from: 'externals/gradle/common.gradle'
apply from: 'externals/gradle/alternatives.gradle'
alternatives.gradle (fragments)
def getDependencySpec(String lib) {
return new Spec<Dependency>() {
boolean isSatisfiedBy(Dependency element) {
String group = element.getGroup() == null ? "" : element.getGroup()
String name = element.getName() == null ? "" : element.getName()
String version = element.getVersion() == null ? "" : element.getVersion()
String dependencyName = "$group:$name:$version"
//println("($dependencyName == $lib) = ${dependencyName == lib}" )
return dependencyName == lib
}
}
}
task createDistributionDir(dependsOn: jar,type : Copy) {
String distDir = "$buildDir/distributions/$baseLine"
from(fileTree(".") {
include "conf/**"
include "lib/**"
})
apply(alternativeLibraries) { entry, group, name, lib ->
Spec<Dependency> dependencySpec = getDependencySpec(lib)
switch (entry) {
case Entry.COMMON:
from(configurations.defaults.files(dependencySpec)) { into "lib/$group/" }
break
case Entry.DEFAULT:
from(configurations.defaults.files(dependencySpec)) { into "lib/$group/$name/" }
break
case Entry.ALTERNATIVE:
from(configurations.alternatives.files(dependencySpec)) { into "lib/$group/$name/" }
break
}
}
}
In common.gradle I have defined:
configurations {
defaults { transitive = false }
alternatives
{ transitive = false }
testRuntime { extendsFrom defaults }
}
sourceSets {
main {
java.srcDirs = ['src']
resources.srcDirs = ['conf']
compileClasspath = compileClasspath + configurations.provided + configurations.defaults
}
test {
java.srcDirs = ['test']
resources.srcDirs = ['test']
compileClasspath = compileClasspath + configurations.provided + configurations.defaults
}
}
Most of my problems were connected with sharing state and function definition between scripts: 1. ext. for writing / for reading 2. proper initialization of ext variables (before or after apply from: …); My intuition was that it is similar to Java import (it doesn’t matter where it is in script), but it’s different. 3. I have another problem with moving out my ‘buildscript’ section of script to another file. This part is common for many projects and should be refactored out from build.gradle. (I found answer in forum that it is currently not possible to move this section out). 4. Also automatic inheritance of configurations seems to be more a problem that gain. Maybe there should be no inheritance, but everything defined manually?
Generally I think that ability to split script into smaller fragments is very important for big projects. And there is a field for improvement in this area for Gradle.
Anyway I was able to solve my initial problem, so I would like to say ‘thanks!’ for your support… 