How to publish multi project build artifacts in a single ivy module?

Old Forum Archive
1

Hello,

I am trying to publish an ivy module based on multi project build. The build consists of three projects core, server, client where client and server depend on core. For argument’s sake let’s assume that publishing core jar on its own doesn’t make sense.

So, I would like to publish this project as a single ivy module with two configurations (client and server). The produced ivy.xml file should look something like:

<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven">
    <info organisation="org.bpp"
          module="ivy-test"
          revision="1.0"
          status="integration"
          publication="20121125200638"
            />
    <configurations>
        <conf name="client" visibility="public"/>
        <conf name="server" visibility="public"/>
    </configurations>
    <publications>
        <artifact name="ivy-test-client" type="jar" ext="jar" conf="client"/>
        <artifact name="ivy-test-server" type="jar" ext="jar" conf="server"/>
        <artifact name="ivy-test-core" type="jar" ext="jar" conf="client,server"/>
    </publications>
    <dependencies>
        <dependency conf="core" org="com.google.guava" name="guava" rev="13.0.1"/>
        <dependency conf="client" org="'org.antlr" name="stringtemplate" rev="4.0.2"/>
        <dependency conf="server" org="com.google.code.gson" name="gson" rev="2.2.2"/>
    </dependencies>
</ivy-module>

This is a how I layout the project:

ivy-test
+--- settings.gradle
+--- build.gradle
+--- core
|
  +--- src ...
|
  \--- build.gradle
+--- client
    |
  \--- build.gradle
|
  +--- src ...
\--- server
     +--- src ...
     \--- build.gradle

I had a few stabs at it - all of them without success. See contents of gradle files at https://gist.github.com/4145778

All in all I would like to do something similar to how ivy.xml looks like for [spring on ivy roundup] (http://ivyroundup.googlecode.com/svn/trunk/repo/modules/org.springframework/spring/3.1.2/ivy.xml), but with fewer artifacts of course. Can you point out where I am going wrong?

Is this even something gradle is trying to support or not?

Thanks, Dalibor

Versioning:

------------------------------------------------------------
Gradle 1.2
------------------------------------------------------------
  Gradle build time: Wednesday, 12 September 2012 10:46:02 o'clock UTC
Groovy: 1.8.6
Ant: Apache Ant(TM) version 1.8.4 compiled on May 22 2012
Ivy: 2.2.0
JVM: 1.7.0_09 (Oracle Corporation 23.5-b02)
OS: Linux 3.2.0-33-generic amd64
2

I think that declaring the artifacts manually as you did in ivy_test_build.gradle should work, but you will also have to declare the dependencies, which will make it more complicated, what did not work with this build script?

Other option would be to create a module that just references those subprojects - you will have to upload all of them as well and the artifacts will still go with the subprojects, but that this matter that much? Yet another option would be to build all the artifacts in a single project (possibly with different sourcesets).

3

Hello Detelin,

Thanks for offering the hints and tips.

As you have rightfully pointed out, one of the problems lies in dependencies (or actually in lack of them) in ivy.xml. As you can see from the [actual ivy.xml produced by gradle] (https://gist.github.com/4145778#file_actual_ivy.xml) there are a few things which are a bit off:

  • ‘archives’ config appears out of thin air for no apparent reason and it actually contains ivy-test-client

  • dependencies are completely missing - I would have to come up with yet another workaround/hack to sort them out

  • I had to repeat myself - I have already declared that both client and server depend on core and I had to declare that again. I have also had to hardcode the paths to aritfacts (this could be done a bit nicer with interpolation, but still noisy). If I wanted to sort out external dependencies I would probably have to repeat config dependencies again. It all seems a bit much considering that this information is already available. This is why I tried to use :client:jar and :server:jar respectively as artifacts to publish (see commented out sections in [ivy_test_build.gradle] (https://gist.github.com/4145778#file_ivy_test_build.gradle)) in hope that artifacts and dependencies might be resolved correctly, but without success.

So I am still wondering if it is possible to organise and publish the module in the way I’ve described. It is an elegant way to manage a module (see [ivy roundup projects] (http://ivyroundup.googlecode.com/svn/trunk/repo/modules.xml)) and I am a bit surprised by how difficult it seems to be to achieve. Especially considering how easy it is to do in the old ‘ant + ivy’ world. Am I missing something?

Thanks, Dali

4

The ‘archives’ configuration appears because you apply the ‘base’ plugin, it contains just ‘ivy-test-client’ artifact due to the current implementation of the DefaultArtifactPublicationSet.

About dependencies - I think there is no repetition after all - if you describe the dependencies of the client and server in their respective build scripts and upload them to the target repository, then you do not have to declare their dependencies once again, but only reference them as project dependencies:

dependencies {

client project(’:client’)

server project(’:server’)

}

On the other hand, if you are not willing to upload the client and server modules, then you will have to re-declare all their dependencies in the grouping project as well (you can still copy the dependency definitions instead of re-declaring them). At the end, there will be no duplication on the repository that you upload to. I’m not sure this is much easier to implement in pure ‘ant + ivy’ - how can this happen if you again want to have separate :client and :server subprojects?

5

Hi,

According to the javadoc in [DefaultArtifactPublicationSet.java] (https://github.com/gradle/gradle/blob/RB_1.3/subprojects/plugins/src/main/groovy/org/gradle/api/internal/plugins/DefaultArtifactPublicationSet.java) it should only be used when configuration is not explicitly set and I do that. It seems that it does not honor this contract and to me the result does indeed look weird. Maybe there is some scope for improvement in this area.

You got me thinking about uploading client and server projects to a repository so that I can reference them later on. Maybe I can just “upload” them to a local file system repository. But it all again starts feeling like a hack/workaround.

What I was really hoping for is some kind of first class support for what I’m trying to do. And it feels like it should be possible to do it as all the needed information is there. As I said before - it feels like I should be able to say something like

artifacts {
    client ':client:jar'
    server ':server:jar'
}

in [ivy_test_build.gradle] (https://gist.github.com/4145778#file_ivy_test_build.gradle) and the rest can be deduced. Can this be done? Should I raise some kind of request? I’ll be up for helping with this task if it is feasible, does not violate any gradle principles and if I can rely on a bit of guidance.

Regarding ‘ant + ivy’: publishing in this manner is trivial as you have full control over ivy.xml. I agree that the build scripts might get a bit hairy - at the moment I rely on a “base” script that allows me to add additional “projects” by just specifying their location and configuration name. There are drawbacks as I can only run commands from the base directory where the single build.xml actually lives, etc. This is one of the reasons why I am trying to move to gradle - I am hoping for a nice build scripting environment (and I have had all my expectations met thus far), but also for that niceness to follow me all the way through the publish phase.

Thanks again for the time and effort, Dali

6

at the moment I rely on a “base” script that allows me to add additional “projects” by just specifying their location and configuration name

Assuming you have same ‘client’ and ‘server’ projects in Ivy, how does your “base” project generate the desired_ivy.xml?

7

Well in ‘ant + ivy’ world I just have one build.xml and one ivy.xml. The “subprojects” don’t have their separate builds. So I don’t need to generate the file - I have to write it and thus I can write it however I want.

8

The same can be implemented in Gradle using the ‘java-base’ plugin and separate ‘core’, ‘client’ and ‘server’ source sets:

apply plugin: 'java-base'
  defaultTasks 'clean', 'build'
version = '1.0.0'
sourceSets {
 core
 client {
  compileClasspath += core.output
  runtimeClasspath += core.output
 }
 server {
  compileClasspath += core.output
  runtimeClasspath += core.output
 }
}
  configurations {
 clientCompile.extendsFrom coreCompile
 clientRuntime {
  extendsFrom coreRuntime
  visible = true
 }
    serverCompile.extendsFrom coreCompile
 serverRuntime {
  extendsFrom coreRuntime
  visible = true
 }
    archives.extendsFrom coreRuntime, clientRuntime, serverRuntime
}
  dependencies {
 coreCompile
 'com.google.guava:guava:13.0.1'
 clientCompile 'org.antlr:stringtemplate:4.0.2'
 serverCompile 'com.google.code.gson:gson:2.2.2'
}
  task coreJar(type: Jar) {
 from sourceSets.core.output
 appendix 'core'
}
  task clientJar(type: Jar) {
 from sourceSets.client.output
 appendix 'client'
}
  task serverJar(type: Jar) {
 from sourceSets.server.output
 appendix 'server'
}
  artifacts {
 coreRuntime
 coreJar
 clientRuntime clientJar
 serverRuntime serverJar
}
  tasks.build.dependsOn 'coreJar', 'clientJar', 'serverJar'
  repositories {
 mavenCentral()
}
9

Hi Detelin,

Thanks for the example of single project with multiple source sets. I am aware of this option and was considering it, but it seems that with it I have to do lots of “manual code writing”. It does indeed mimic what I have to do currently in ‘ant + ivy’, but I was hoping for something a bit more elegant. I guess my expectations might have been too/unreasonably high.

I see that there is an [ivy-publish plugin] (http://www.gradle.org/docs/current/userguide/publishing_ivy.html) on the books now. I’ll have a look at that as well.

Thanks for your time and patience, Dali