Configure different set of repositories based on project status

Old Forum Archive
1

Hello everyone,

We have setup 3 different Ivy repositories for upload/publishing corresponding to the three different maturity levels that a project can have: - releases - contains modules with ‘release’ status - milestones - contains modules with ‘milestone’ status - snapshots - contains modules with ‘integration’ status

So if a projects sets ‘status=milestone’ and the ‘upload’ task is executed, it is published to the ‘milestones’ repository.

I have been thinking to implement similar functionality for resolve, i.e. if a project sets ‘status=milestone’, it shall be allowed to resolve dependencies only from the ‘milestones’ and ‘releases’ repositories.

This would work fine for project dependencies but I’m wondering how it could be implemented for buildscript dependencies, since at the moment the buildscript’s classpath is constructed, the project status might not be known. At the moment we are setting the project’s status in a custom plugin based on certain other properties of the project (e.g. its version). This custom plugin must of course be defined in the buildscript’s classpath dependencies so it cannot be executed before the buildscript’s repositories are configured and the buildscript classpath is resolved. Does anyone has any suggestions how to solve this?

2

What’s the core problem here?

Is it that ‘release’ projects can only use ‘release’ build plugins?

3

The problem is that in order to enforce this, we have been using a custom plugin, but the plugin itself must be resolved from the buildscript’s repositories so we cannot enforce this for the buildscript’s dependencies - since the plugin is available only after they are resolved, it’s like the chicken-and-the-egg problem. Probably we can add the logic that enforces this for the buildscript’s dependencies directly in the Gradle init script.

4

Enforce what exactly?

It sounds like you are trying to tie the lifecycle of the build plugins to the lifecycle of the component being built. Is this the case?

5

I have the same setup (different repositories for builds of differing maturities) except using maven repositories. In my case, the maturity of the project is independent of the maturity of the build itself as the build plugin is just another project built by my build. Essentially all builds, irrespective of their maturity, are built by “final” build plugins only (unless you’re testing the build plugins).

I think the q is why do you want the build plugin to be dependent on the thing you are building?

6

I will try to explain what I want to do with an example:

First, let’s assume that we are using Ivy metadata and we chose a version syntax that:

  • does not have a notion of a changing revision (aka SNAPSHOT) - does not have any maturity information in the version (e.g. milestone, rc, etc.)

Then we end up with something like ‘major.minor.micro-timestamp’ for a non-release version (integration or milestone), e.g. ‘1.0.0-20121001’. However since Ivy’s latest revision strategy would consider this version greater than the ‘1.0.0’ release version, we have to add some character before the timestamp, which would lead us to a proper ordering:




 1.0.0-v20121001 < 1.0.0
We then define three repositories for each maturity level (status of a project):
  • 'release’s - contains modules with versions e.g. ‘1.0.0’, having release status - ‘milestones’ - contains modules with versions e.g. ‘1.0.0-v20121001’, having milestone status - ‘snapshots’ - contains modules with versions e.g. ‘1.0.0-v20121101’, having integration status

Now if I want to enforce that a project with a integration status can resolve from ‘releases’ and ‘milestones’ repositories only, I would have to write a plugin to do that, let’s name it ‘releaseEnforcer’

However, I cannot restrict the users to use only the"final" (release) version of this plugin - I consider the buildscript’s dependencies like any other dependency so if the project is having a milestone status, then it should be allowed to resolve milestone buildscript dependencies as well. I cannot force the users what version to use when referencing this plugin, they could even use a version range, e.g.

buildscript {

dependencies {

classpath ‘org.foo:releaseEnforcer:[1, 2)’

}

} But if they do the above, since the plugin cannot run before it is resolved, the buildscript’s repositories would still contain all resolvers (‘releases’, ‘milestone’ and ‘snapshots’), so the build might pick up e.g. ‘1.0.0-v20121101’ version which is an ‘integration’ version.

What I want to achieve is that the build picks up ‘1.0.0-v20121001’ (a milestone) if the status of the project is also ‘milestone’ (or release) - here I assume that the status of the project can be identified before evaluating the build script (e.g. by examining the task list or project properties) - to achieve this goal the ‘releaseEnforcer’ must remove the ‘snapshots’ repository for the buildscript’s and the project’s repositories - thus any higher integration version there will be ignored.

I guess the solution would be to add the logic that restricts the set of repositories to resolve from in a Gradle init script instead of adding it to a custom plugin, this way it could run even before the buildscript’s dependencies are resolved. The only other thing to ensure is that the project’s status cannot be changed after that (e.g. during the evaluation of the build script).

I probably answered to my question myself, but I wonder whether all this sounds reasonable or I’m making wrong assumptions?

7

I consider the buildscript’s dependencies like any other dependency so if the project is having a milestone status, then it should be allowed to resolve milestone buildscript dependencies as well.

This is a rather unusual approach. These two bits of software usually do not share the same lifecycle.

here I assume that the status of the project can be identified before evaluating the build script

The only way to do this is to move the status of the project outside of the project. You cannot reference anything about the project in a meaningful way from the buildscript block. This is a chicken/egg problem.

I probably answered to my question myself, but I wonder whether all this sounds reasonable or I’m making wrong assumptions?

You’ve got a handle on it I think. I’d have a second think on whether you really want to tie the build plugins to the lifecycle of the code under development. I don’t think it’s a good idea to couple them in this way.

What I have seen before are checks to ensure that final production builds do not build with non final plugins. That’s quite reasonable.

8

Thanks Luke,

Regarding the build plugins - we are actually building milestones of our build plugins every 2 or 3 weeks or so, infact we do not have a release version yet. So products are usually declaring the ‘latest.milestone’ of the build plugins’ module in their buildscript’s dependencies. The build plugins’ project is having similar lifecycle as the other projects - everyone is building milestones, but the build plugins’ project is expected to produce a release version before everyone else (so that they could switch to it before releasing). At least that’s the situation I’m dealing with.

Based on your feedback, I think we can have some really small module, let’s call it ‘build-init’ that is declared as a dependency of the Gradle init script - its purpose would be to identify the project status (based on project’s properties and/or start parameter tasks) and then restrict the set of repositories for the project (as well as for the buildscript) to those having at least the maturity level of the project. The ‘build-init’ module would change rarely and the Gradle init script will depend on a release version of it (not a milestone or snapshot). But still, each project might decide to use a milestone or snapshot of the standard build plugins that we are developing, e.g. if they want to test some new feature or bugfix.

One thing I would like to achieve is to use the ‘build-init’ code to plug-in some convention objects into each project after it is loaded and to reuse those objects from our build plugins. For example, at the moment we have a ‘base’ plugin which reads a couple of project properties like ‘build.number’, ‘fix.number’, ‘build.version’, etc. and initializes a custom convention object that encapsulates those. Now I would like to do move all this initialization in the Gradle init script - hopefully I would be able to add convention objects at that time and reuse them later in our build plugins after the project is evaluated.