Using gradle in continuous deployment


(bbeaudreault) #1

[Thanks to Jason Porter for pointing me here from the user group]

Hello all,

We use continuous deployment at my company in its truest form: everyone is always committing and deploying, all the time. We have a bunch (200+) of java projects, many linked in some way as a dependency. E.G. We have may project A, which depends on B and C. C may also depend on D, while B depends on E. All of these may have external dependencies too.

We have been using a custom built deploy script, mixed with Maven for this dependency management. For normal external dependencies it checks the maven2 central repositories. For internal dependencies, our deploy script parses the pom.xml (and pom.xml of all dependencies recursively) and checks out the source for each from SVN. It then does maven install/build on each of those, until we can finally build end result, project A. When we deploy, we always check out the latest version of each of our internal dependencies.

In practice this isn’t as horrible as it may sound, but I’d like to make it better. Gradle looks really interesting, but I’m not sure it supports our needs:

Basically we need to be able to specify dependencies that are external (maven2 repo), but also be able to specify internal ones by pointing to a particular project name or source control repo. Ideally it would work on both svn and git. Example (not in proper gradle syntax, since I haven’t yet fully learned it):

project A {

dependency ‘b’ ‘https://mysvnrepo.com/svn/someProjectB

dependency ‘c’ https://myGITrepo.com/git/someProjectC

dependency ‘d’ mavenRepository() }

Each of those dependencies B and C could have further dependencies, and each would have a build.gradle/settings.gradle so that we could check out recursively. It is important that the build does not specify a specific version, and always pulls HEAD from the repo and builds that. We have no notion of version numbers, like I said.

Is this something gradle can support (now or in the future), or do you know of something that can?

I noticed the multi-project mode of gradle, but I’m not sure it supports this either. While we are virtually set up like your example, with a base folder and a bunch of sub projects, we never ever are deploying all of them at once. It seems like the current multi-project mode would require you to check out the entire repository (all 200+ “subprojects”), under a base project to do the deploy. We may only actually need like 3-10 of those, and checking them all out is time consuming.

Thoughts?

Thanks,

Bryan


(Szczepan Faber) #2

Hey Bryan,

Although your setup is pretty unusual I think it should be implementable using gradle. With gradle, you have the power of groovy to customize your build needs, e.g. do the git clone on demand, etc. So if you already have a custom build in language x (I’m betting it’s ant :slight_smile: it should be possible to work it out using Gradle, taking advantage of some Gradle little gems :slight_smile:

I’ve seen a team that uses their very own formatted file for declaring the dependencies. So I think your requirement can be met.

You’re right, gradle multimodule vanilla setup won’t work for you. You would have to customize the builds a bit using groovy so that the settings.gradle contains selected projects rather than all 300. I’ve seen teams using settings.gradle that contained some smart logic to select the projects.

One thing you should watch out is that gradle multimodule build does not support nested ‘settings.gradle’. This means that a ‘single setup’ for a team would be a multimodule gradle build with single ‘settings.gradle’.

In future we plan to add some more features regarding build aggregation so that it should be very easy to work on a subset of projects rather than working on the entire multimodule project.

Hope that helps!


(Luke Daley) #3

Hi Bryan,

What you want to achieve is definitely possible, but Gradle doesn’t have built in support for that kind of workflow out of the box. But as Szczepan said, one of Gradles virtues is that it allows you to very conveniently plugin your own build logic in a way that makes sense for your domain.

You’d generally follow a similar process, just moving most of your already developed logic into Gradle tasks. You wouldn’t make it a multi project build. Instead you’d add some plugins that allow you to express your internal dependencies in the build file, and then use this information to traverse through the projects installing them into a local repository (like you do now).

Since you have all this already, you might wonder why you would move it to Gradle. The biggest benefit I can see without knowing too much about your project is that you would then have access to all of the other things that Grade can do (e.g. powerful file operations, convenient process execution etc.) that you’d probably have to code up yourself with your in house solution. In short, you’d be porting a lot of your developed workflow into Gradle and getting access to all of the other Gradle features in return.


(bbeaudreault) #4

Hey guys,

Thanks for the thoughtful responses. It looks like Gradle is pretty flexible :). I’ll think on this info and decide if it is worth the investment to port over our current logic.

Thanks again,

Bryan