Complex configuration vs. runtime use case - how to solve?


(Mike Meessen) #1

Hi there,

Recently, I came across a pretty complex issue to be solved in one of our projects. Here’s the deal: we deliver an application that runs in an apache servicemix osgi container. The problem is that, by default, this container downloads a bunch of jars off the internet when applications and features are first deployed. This, of course, doesn’t work on a production server. Therefor, the idea is to resolve every required jar at build time and package everything up into the delivered application package. That’s the easy part: we define a dedicated configuration, put in all dependencies required by our application that don’t come shipped with the servicemix container and use a simple copy task to include this configuration’s artifacts into our application package like so:

configurations {
    servicemix
}
  dependencies {
    servicemix (group: 'commons-dbcp', name: 'commons-dbcp', version: commonsDbcpVersion) { transitive = false }
    servicemix (group: 'oracle.jdbc', name: 'ojdbc6', version: oracleDriverVersion)
    ...
}
  task assembleServicemix(type: Copy) {
    ext.targetDir = "${buildDir}/servicemix/"
      configurations.servicemix.resolvedConfiguration.resolvedArtifacts.each { dep ->
        def id = dep.moduleVersion.id
        from(dep.file) {
            into "/system/${id.group.replace('.','/')}/${id.name}/${id.version}/"
        }
    }
      into targetDir
}

Now for the added complexity: besides our application’s own dependencies, the servicemix container also needs to download a couple of its own dependencies (yes, it comes incomplete out-of-the box…) when some of its “features” are being deployed for the first time. So we need to also include the jars of these “features” in our application package in order for them NOT to be downloaded in production. That shouldn’t be too hard, we’d just need to add them to the “servicemix” configuration above, right? Well, not so simple. Here’s the thing: we’re not talking mvn bundles here. We’re talking “servicemix feature names” who somehow end up in a list of mvn artifacts… a lot of them! You certainly don’t want to “resolve” them by hand, and we want to be able to switch to a new servicemix version without having to re-resolve by hand again here.

Now, there IS a way to resolve “feature-name => list of required mvn bundles” in a mechanical way. So I implemented that in a couple of groovy lines which, eventually, add the dependency in the “servicemix” configuration dynamically. However, resolving “feature-name => list of required mvn bundles” involves crawling some servicemix files, which in turn involves downloading and unpacking a servicemix base image. And finally, all of that is way too much to be done in the gradle configuration phase!

Here’s how the dependencies get dynamically added in a dedicated task:

task extractDependenciesFromFeatures() {
    doFirst {
        // Do loooots of stuff here...
          // Finally, add all bundles to the servicemix configuration dynamically
        targetBundles.each {
            project.dependencies.add("servicemix", "${it.group}:${it.artifact}:${it.version}")
        }
    }
}

I guess, lifecycle-wise, the only viable option is to put everything (servicemix download, servicemix unpacking and feature-to-bundle resolution) in own tasks and set up the dependencies between them correctly.

Now the issue kicks in when I go back to the initial copy task, which traverses the dependencies from the “servicemix” configuration for them to be copied. As it stands here, that’s being done in configuration phase, which is too early, since the feature resolution hasn’t been done here yet.

I’m a bit stuck now.

One option is to modify the copy task to add the dependencies from the configuration at run time like so:

task assembleServicemix(type: Copy, dependsOn: ['extractDependenciesFromFeatures']) {
    ext.targetDir = "${buildDir}/servicemix/"
      // Always clean to force running
    delete(targetDir)
      into targetDir
      doFirst {
        configurations.servicemix.resolvedConfiguration.resolvedArtifacts.each { dep ->
            def id = dep.moduleVersion.id
            from(dep.file) {
                into "/system/${id.group.replace('.','/')}/${id.name}/${id.version}/"
            }
        }
    }
}

But as you can see, that -of course- breaks the contract for gradle’s up-to-date check (which evaluates during configuration phase) and forces me to clean targetDir on every run (during configuration)! … which is dead wrong and ugly.

I guess because of all the dynamicness here, I’m stuck somewhere between the static configuration-time and the dynamic run-time. Any ideas / advice of where I went wrong, or how I can get out of here?

Thanks in advance, Mike


(Luke Daley) #2

You can use a configure task for this…

task configureCopy << {
  copyStuff {
    // do expensive configuration of copyStuff task here
  }
}
  task copyStuff(type: Copy, dependsOn: configureCopy) {
    }

The one caveat is that you can’t do anything in configureCopy that introduces new task dependencies.