Organization help!

Help/Discuss
1

I am trying to figure out how best to layout our projects in order to share common build logic.

We are currently laid out as follows…

core
core-feature-a
core-feature-b
etc...

Each of the above are separate github repositories and gradle projects

Then we client implementations…

clientImpl-A  (dependencies - core, core-feature-A)
clientImpl-B  (dependencies - core)
clientImpl-C  (dependencies - core, core-feature-A, core-feature-B)
etc...

each client impl is a separate github repo and gradle project and includes the “core-…” repos.

Settings.gradle ex.

include ":core"
project(":core").projectDir = file("/path/to/core/repo")

The problem is that we are duplicating build logic in each of the client impls. It would be ideal if we could include common build logic in “core” repo buildSrc, but I don’t think gradle can do this.

What are some ways we can consolidate the duplicated code?

Are my only options to use a maven repo and publish a plugin? Or to structure all of the repos under a common gradle project similar to https://github.com/spring-projects/spring-framework?

I’d prefer not to use the spring approach, as it doesn’t seem appropriate

2

How are the separate projects currently depending on each other? I’m guessing you are publishing artifacts to a maven repo. If that’s the case thent I suggest that you also publish a gradle plugin to the same maven repo and reference via the GAV in other projects. As you suggested another option is a single multi project build but that’s usually only suitable for projects with a common release cycle.

Working locally you can publish to maven local or use a composite build but the “enterprise” solution is to publish to a repository

3

We don’t have a maven repo. Currently the projects are all based off of the
master git branch. The dependency is:

    compile project(":core")
4

Each of the above are separate github

This statement says to me that each project is a separate build

compile project(“:core”)

This statement says that it’s a single multi-project build

I’m confused :confused:

5

That’s why i’m looking for organizational help :slight_smile:

so each “core” project is a separate build, however each clientImpl includes the “core” project as a multi-project build

so…

clientImpl-A is a multi-project build consisting of clientImpl-A, core, and core-feature-A
clientImpl-B is a multi-project build consisting of clientImpl-B and core

how would you recommend organizing the projects?

We currently don’t do versioning, but this is part of the process of implementing versioning

6

Are you duplicating the “core” sources in both projects? Or do you have multi module builds that span two (or more) source repositories (eg 1 build for 2 git repos). I don’t recommend either of these approaches.

I’d recommend that projects that are versioned together and released together are in a multi module build (and all in the one source repository). Projects with separate release cycles should have separate builds and should live in separate source repositories. A single build should not cover 2 or more source repositories (unless local development via composite build)

A continuous integration server (eg Hudson/Jenkins/Travis) should constantly be building all projects and publishing snapshots to a maven repository. Dependencies for CI builds will also be sourced from the maven repo. Some people recommend never using “-SNAPSHOT” versions (eg use git hash instead) but I’ve never done this myself.

When building locally you will either source dependencies from the maven repo (if not changing said dependencies). If also changing the dependencies locally you can develop two projects in parallel using composite builds or maven local repository.

7

So I’m finally getting around to this.

A question…

is it possible to dynamically remove an included build? I would like to force the use a jar files for certain tasks that are run

8

for anyone interested…

I was not able to do this using the gradle composite build, however I was able to implement this a modified version of the CompositePlugin in this post.

I modified the resolveDependency function to use gradle’s dependencySubstitution as follows:

if (!forceJars && projectPath) {

    Project childProject
    try {
        childProject = project.project(":${projectPath}")
    } catch (UnknownProjectException e) {
        // No local project, use maven repository for dependency resolution. This is not an error condition but an expect
        childProject = null
    }

    if (childProject) {
        project.configurations.each { cfg ->
            cfg.resolutionStrategy.dependencySubstitution {
                substitute it.module(dependencyNotation) with it.project(childProject.path)
            }
        }
        if (log.isTraceEnabled()) {
            childProject.tasks.each { log.trace(it) }
        }
        log.quiet("Found non-jar project dependency: " + childProject)
        project.composite.resolvedProject = true
    }
}

Then you can add another substitutionRule to replace the project with the module:

project.configurations.each { cfg ->
    cfg.resolutionStrategy.dependencySubstitution.all {
        if (it.target instanceof ProjectComponentSelector) {
            it.useTarget it.oldRequested.toString()
        }
    }
}
1 Like
9

BTW, most of the time we no longer use the “forceJars” property because it affects all builds and dependencies. Instead we just temporarily comment out the appropriate includeFlatIfLocal lines in settings.gradle.