Switching between Artifact and Project dependencies?

Old Forum Archive
1

I am trying to set up a multi-project build, and I was hoping to set it up with maven-like build isolation, where each subproject (artifact builder) can be built without requiring any of its dependent code. Basically, I want to swap between using the jar/artifact dependency to using project dependencies when the user requests it.

I am starting to feel like I’m swimming upstream, so I figured it would be a good time to stop and ask.

The goal is, if you have the following file layout

parentDir\
  ProjA\
    build.gradle
    src\
  ProjB
     build.gradle
     src\
  ProjC\
     build.gradle
     src\
  settings.gradle
  build.gradle

With a simple dependency graph of ProjC depending on ProjB depending on ProjA (And all three would depend on parent). It would be preferred when you build ProjC, it uses artifact (downloaded jar) versions of ProjB/A, unless the user indicates they would prefer upstream projects be built.

The reason for this is both convenience and build isolation: A developer working on ProjC shouldn’t necessary have to check out the whole code tree and build if they only care about their downstream projects. There may also be build setup (such as installing compilers/programs onto the build machine) which may be unnecessary for the downstream projects. No need to cause a downstream java developer to download a C++ compiler if they are only touching the java code.

As far as I can tell, there are two possible approaches

  1. Use the maven plugin and terminology, Each build would be independent (no cross-subproject dependencies). In order to build multiple projects, user would have to visit the upstream project, build/install it, and then visit the downstream project. To return to using the server, the user would have to clean her mvn cache.

1a) Do the same thing, but without using the maven plugin. Have the gradle builds be independent and publish to some sort of local cache. 2) Have the build only include the current project in the multi-project build by default, give the user the ability to pass in a property with the names of additional projects to compile, do some magic in the settings.gradle file to read that property and include the projects. Have a check at the artifact dependency section that only adds the dependencies if they were not passed in as compile-time projects by the user.

Since both of those feel a bit like hacking the build system to do something it doesn’t want to do, I was wondering if there is a more conventional way to achieve the effect?

I actually don’t require each of my projects to be completely independent, as it is in the maven-esque model, but there are certain logically components that really should be parts of separate builds. However, there will be some devs working cross-project who will want to build the entire codebase, and I also need to ensure their experience is not confusing or overly verbose. There doesn’t seem to be an easy way to invoke sequential separate gradle builds…

2

Dynamically switching between project and external dependencies isn’t currently a first-class feature, but it will become one in the future. Meanwhile, some users script it for themselves, similar to how you described. Others check out the whole build from source control, relying on Gradle’s incremental build feature to only rebuild the code they are currently working on.

To invoke a build from another build, use a ‘GradleBuild’ task.

3

Is there a ticket/request open for this feature ? This actually stops us to switch from maven to gradle.

The maven way is to use the build workspace as a maven repository and to resolve there fist. So when building you have nothing to tweak in order to link against your local dev version.

I really like the gradle way but I do not see the point of project dependency. A project can be resolved as group:name:version no matter if it is a local project or a binary artifact.

Hoping to see some moves in that direction.

David.

4

One issue, though, with using group:name:version in local projects is the need for needing to update version (and group in a lesser degree) when it changes. I have never liked how you need to change all POMs in Maven when you’re creating a new version.

-Stig

5

I am trying to script this same thing with no luck.

My simplified structure looks like this (but in reality I need to model this same thing many times)

parentDir\
  ProjA\
    settings.gradle
    build.gradle
    src\
  ProjB
     settings.gradle
     build.gradle
     src\

ProjA should be dendent on ProjB.

ProjA settings.gradle

// make the root project up one dir
rootProject.projectDir = new File('../')
  //set the dir for the dependent service
include ':ProjB'
project( ':ProjB' ).projectDir = new File('ProjB' )

ProjA build.gradle snippet

dependencies {
     compile project(':ProjB') //this is line
77
}

I get the following error:

FAILURE: Build failed with an exception.
  * Where:
Build file 'ProjA\build.gradle' line: 77
  * What went wrong:
A problem occurred evaluating root project 'ProjA'.
> Project with path ':ProjB' could not be found in root project ProjB'.

What am I doing wrong. I am trying to model my solution after this example: http://stackoverflow.com/questions/14729072/having-difficulty-setting-up-gradle-multiproject-build-for-existing-repository-l

But I woudl be open to any other solutions…

The solution should mean that ProjA depends on ProjB, but ProjB does NOT depend on ProjA. Further the projects should not be coupled in any other way (e.g. having to put everthing in a setting.gradle in the parentDir.

Thanks :slight_smile:

6

Correction: The error said:

Project with path ‘:ProjB’ could not be found in root project ProjA’

7

Here is a proof of concept: https://github.com/pniederw/elastic-deps

8

This is great. Almost exactly what I need (and probably a lot of other people). However, is there a way to do this that 1) does not edit the subprojects and 2) does not require the subprojects to know about the parent project? Given the way our team works, I need to create an aggregator project that none of the subprojects know exists. I’ve been trying to accomplish a decoupled approach like this for the past two hours with no luck.

9

No, there isn’t. There is no such thing as “an aggregator project that none of the subprojects know exists”, except if you maintain totally separate builds (which I’d strongly advise against). Instead I recommend to author one build that is flexible enough to meet the expectations of all users.

10

We make hundreds of apps. Our subprojects are common libraries (think: networking, common-widets, utils, things like that). So when we create a new Android app, we compose it of several of smaller pieces. Those pieces need to be able to be built in lots of different apps and in some of those, the developers will be expanding those subprojects (like, adding to utils) so they want that code checked out and their changes need to easily be applied to their local builds. Certain devs prefer doing things “by hand” and don’t want to deal with aggregators. Others, like me, want to automate/simplify everything. For each app, I like one parent project to control everything (from git cloning, to gradle building, to IDE config). So it feels like we must maintain separate builds. Each project needs to stand alone, decoupled from all others and each “aggregator” needs to be self-contained. I appreciate all your help today and would love to follow your advice but I can’t think of how to make “one build that’s flexible enough” to meet all our requirements. Using the “elastic dependency” approach doesn’t work because some people will not check out the aggregators, period. What’s the best workaround for that?

11

I decided to convert this “comment” to a “topic” since it’s related to the main thread. In response to @Peter on creating “one build that is flexible enough to meet the expectations of all users”:

We make hundreds of apps. Our subprojects are common libraries (think: networking, common-widets, utils, things like that). So when we create a new Android app, we compose it of several smaller pieces, which can belong to many different apps.

Certain devs prefer to check out only one subproject at a time and don’t want to deal with aggregators/parent projects. They just build their current subproject, which pulls its dependencies from our maven repo. If/when they edit dependencies (like adding a new common widget), they create a new workspace and open a new IDE window and work on that subproject, whose build is self-contained.

Others, like me, want to automate/simplify everything. GIT projects are cheap to make so for each app, I like to make a new parent project to control everything that I will need to do (from GIT cloning, to gradle building, to IDE config). I import this one project into Android Studio and any code I need to touch, across about 8 subprojects, is at my fingertips and ready to be included in the next APK when I build.

So it feels like we must maintain separate builds. In practice, it seems like each project needs to stand alone, decoupled from all others and each “aggregator” needs to be self-contained. And like the OP, we need to be able to “switch between artifact and project dependencies.” I appreciate all your help today and would love to follow your advice but I can’t think of how to make “one build that’s flexible enough” to meet all our requirements. Using the “elastic dependency” approach doesn’t work because some people will not check out the aggregators (i.e. parent projects), period. What’s the best workaround for that? A plugin? Also, I think the dynamic settings file and the “elastic” method will confuse the IDE and produce many false errors.

12

I have a need for this too. I was able to figure it out. Here is my solution:

https://gist.github.com/vangorra/c1383c355ce8fe56adf8

13

Have you looked at https://github.com/prezi/pride? Seems to support some of the goals.