Multi-project build - project dependencies and build order

(David niquette) #1

I need to migrate a multi-project build from maven to gradle and maintain the way inter-project dependencies and build order work. I’d like to use the maven plugin in gradle and continue to publish artifacts to both local and remote maven repositories.

The multi-project structure is like so:

In maven Project-A2 has a dependency on Project-A1. If I run mvn install_ from the Project-A2 directory it will only build/install that project and pull it’s dependency on Project-A1 from the local/remote maven repository. If I run mvn install from Project-A it will build/install both Project-A1 and A2 and calculate the build order based on the above mentioned dependency. How can this same behavior be achieved in gradle?

Additionally, Project-B2 has a dependency on Project-A2. If I run mvn install from the Project-B2 or Project-B directories this dependency should be pulled from the local/remote maven repository. If I run mvn install from the root directory it should calculate the build order such that Project-A1 builds, _Project-A2 builds, and then Project-B2 builds.

(Stefan Oehme) #2

Could you explain why you want to mimic this behavior in Gradle? Why not build Project-A1? Gradle has excellent support for incremental builds, so this will be fast and correct.

(David niquette) #3

The main concern is build time. If it were simply one small project depending on another it’s not really an issue. I’m working with a number of cases where a relatively small project depends on one or more larger projects. If the project I want to build takes 5 minutes but it’s dependencies take 30 minutes it’s not ideal to build both and end up with a 35 minute build time.

The incremental build support is great and after building everything once it looks like it should address this need. However I do need to consider the case where someone checks out the source fresh and only wants to build a small piece. They’re not going to want to wait for multiple dependencies to build when they could be pulled in from a repository.

(Stefan Oehme) #4

Are these real numbers or hypothetical? Note that the upstream dependencies only need to create their outputs. They won’t run tests or other time-consuming tasks.

If your projects actually take that long just to compile, then you might be interested in either:

  • breaking your big build into smaller builds (inside the same repo) using Composite Builds
  • our upcoming distributed cache support

(David niquette) #5

Those numbers are hypothetical but our total build time does reach between 2-3 hours without test and there are cases that exist similar to that example.

I’ll take a look at composite builds and the distributed cache support. It sounds like a little restructuring of the projects/process to better fit gradle is the direction to go. Thanks for the suggestions!

(Stefan Oehme) #6

Maybe I misunderstood your description: Is the 2-3h build time with Maven or Gradle? Gradle is much faster than Maven, so I’d suggest first going with the straightforward multi-project approach and only optimizing if you actually hit a problem.

(David niquette) #7

2-3 hours in maven. I don’t have an estimate for gradle yet due to trying to work out the best approach ahead of converting the entire maven build to gradle. I’ve only been working with a small piece of our source with gradle.

If the best way to tackle this is to setup the multi-project and then look into composite builds, distributed cache support etc. then that’s what I’ll do. I assumed that we would see similar build time/dependency related problems in gradle as we do in maven but if that’s not the case that’s great. Thanks!