Referencing local dependency projects without publishToMavenLocal step

Developer Productivity Engineering (DPE)
1

I was wondering if anything can be done to streamline the developer experience of working with shared libraries in consuming applications.

I basically have this same exact scenario:

That is, I maintain a collection of internal libraries for my company that are consumed by many applications. When working on one of the libraries, and absorbing those changes into an application, my current workflow is something like this:

common-library-1 $ 
./gradlew publishToMavenLocal
# Take note of published snapshot version

common-library-2 $ 
# Edit gradle.properties to bump common-library-1 snapshot
./gradlew publishToMavenLocal

consuming-application $ 
# Edit gradle.properties to bump common-library-2 snapshot
./gradlew check

Could library-1 and library-2 be in a monorepo as modules? Maybe. And this would save one step. But library-1 is Spring Web MVC and library-2 is WebFlux. Both are themselves multi-module projects. So the separation feels warranted.

It would save me significant time and tedium if Gradle offered the equivalent of npm link; a way to tell it:

“There is a dependency right next door in this sibling directory, dear Gradle. Do not require me to go build or publish it for you. That’s literally your job. It’s right there. You can do it. I believe in you.”

Something like:

dependencies {
  api localWorkingCopy('../common-library-1')
}

Or, in a way that could be versioned:

// settings.gradle
repositories {
  localWorkingCopy {
    'com.mycompany:common-library-1' dir('../common-library1')
  }
}

// build.gradle
dependencies {
  // will fall back on default behavior if localWorkingCopy directory is not found
  api 'com.mycompany:common-library-1:1.2.3' 
}

Something to that effect. Has this been considered before?

2

Yes, and it is there for ages.
What you are after are composite builds.
You can simply includeBuild the libraries in the consumer and if they are not doing bad stuff (like manipulating the coordinates on the publication) and have the same (or at least a compatible Gradle version) then the artifacts provided by the included build are automatically replaced by an on-demand built version and you even get all the projects opened together in the IDE.

The “bad stuff” can be mitigated by manual dependency substitution when including the build, but better fix the included build to be more idiomatic so that it just works.

A non-compatible Gradle version cannot be mitigated currently as all included builds are built with the Gradle version of the including build.

Btw. if you use mavenLocal() be aware of the many problems you get with it and at least make sure it is always the last repository in the list and optimally always with a repository content filter that defines exactly what should be resolved from there. Otherwise you mainly earn slow and flaky or even silently misbehaving builds.

1 Like
3

Eeeenteresting. I’m using includeBuild in common-library-1 for some custom Gradle plugins, but thats been the extent of my using it.

That’s not something I could version, right? The workflow would be:

  • If I’m working on the common-libraries locally, edit the application’s settings.gradle to “toggle” to the local / includeBuild block
  • And also edit the application’s build.gradle to comment out the dependencies entries for the common-libraries?

Does that sound right? I’ll give that a shot.

And thank you for the tips about mavenLocal(). I was not aware it had so many caveats.

4

That’s not something I could version, right?

Sure you can.

Well, if you use the commandline argument or IDE integration to define the composite build, you cannot, unless you modify the wrapper start script.

But the includeBuild in your settings script are versioned, aren’t they?
And you have a turing-complete language at hand in the settings script, so you could also do things like checking whether “$settingsDirectory/../common-library-1” is a directory and if so do an includeBuild on it or similar fun stuff.

1 Like
5
  • And also edit the application’s build.gradle to comment out the dependencies entries for the common-libraries?

No, as I said, that is not necessary.
Your use-case is exactly the original use-case for composite builds.
If you would comment out the dependencies, you would not have a dependency.
If the dependency declared like you have it now is found in an included build, it is taken from the included build instead, version is ignored.

1 Like
6

And thank you for the tips about mavenLocal(). I was not aware it had so many caveats.

The case against mavenLocal()

1 Like
7

Wow. Conditionalized includeBuildworked like a charm, first try. This is a game changer for my daily workflow. Thank you!

// settings.gradle
rootProject.name = 'my-application'

if (new File('../my-core-library').isDirectory()) {
    includeBuild '../my-core-library'
}

if (new File('../my-webflux-library').isDirectory()) {
    includeBuild '../my-webflux-library'
}

include 'api'
include 'application'
include 'terraform'
// ..
1 Like