Plugin to skip publishing if artifact already exists in repository

Help/Discuss
1

Hi everyone

I’m trying to create gradle plugin, which enhances publication tasks in a project so that they first check is apt artifact already exist in repository, and if it does exist – skip publish it again.

It seems to me this is not a rare need, and an easy to do anyway, but I wasn’t able to find anything similar, and still see no easy ways to implement it myself – so I probably missed something obvious here.

Background: our repository is configured to prohibit overwriting release artifacts – i.e. repository rejects PUTs for existent artifacts. Which is pretty common configuration, I believe. But it makes build not idempotent: one can’t just repeat same build from the same version with the same parameters.

So if build pipeline fails in the middle by a side reason one can’t just restart it from beginning, and hope it skips/re-build that was already built, and continue building that wasn’t built yet – instead build process will fail trying to publish the very first artifact that happens to be already published. Which makes people sad.

So I’m looking into ways to implement such conditionally-publish plugin:

  • The most straightforward way seems to intercept the http PUT itself, since at this point everything is already prepared (artifact url, repository with apt auth…). But PUT is done quite deep down the rabbit hole: somewhere around org.gradle.api.publish.maven.internal.publisher.AbstractMavenPublisher.
    It is not clear to me how plugin could intercept here, and is it even possible: seems like plugin evaluation is too late build phase to adjust anything in publication service itself?

  • More conventional way is for plugin to find all PublishToMavenRepository tasks, and enhance them with something like

    doFirst { 
    if( head( artifactURL ) == 200 ){ 
        throw new StopExecutionException("already exists)  
    }
}

    But here again is not clear how to use gradle services to make artifactURL, and access repository (with, probably, auth). I.e. all this could be implemented from scratch, but it seems huge DRY violation, since gradle already implements all this inside, I just don’t see how to access it.

    (Another issue here is existence of legacy Upload task, which also needs to be adjusted, but in a different way)

Is there some clearer way to implement such a plugin? Or, maybe, it is already exists?

1 Like
2

Hi @Cheremin, did you end up producing a solution for this problem?

Cheers

3

We have a use case where we would like to skip not only the publish, but also the build task, if the artifact already exists in the repository. Our build versioning is based on the Git commit, and knows whether the project directory is “dirty” - so if the versioned artifact exists in the repository there is no need to build it. I would be curious to see if anyone has come up with something.

4

You probably need to register an onlyIf action for the relevant tasks that requests from the repository whether the version is already published or not to achieve that. Or actually if you need it for multiple tasks, but do not want to evaluate it in configuration phase, then maybe you should have the logic in a shared build service you then query in the onlyIf action.