Confusion over plugin dependencies


(Steve Cohen) #1

I have been using Gradle for over a year, and my practice has used local maven repositories a lot.

Now I am introducing the system to our team and finding unexpected difficulties when trying to do local builds.

I have a project with a top level buildscript that looks like this:

    buildscript {
        repositories {
            jcenter()
            mavenCentral name: "nexus", artifactUrls: [
                "http://mavencentral.it.whatever.com:8084/nexus/content/groups/whatever-public-group",
                "http://mavencentral.it.whatever.com:8084/nexus/content/repositories/whatever-repository-releases"
            ]
            maven {
                url "https://plugins.gradle.org/m2/"
            }
            mavenLocal()
         }
          
         dependencies {
             classpath group: 'com.whatever.subgroup:rpm_gradle:1.1.1'
         }
    }

rpm_gradle is a collection of plugins that I have built. This plugin package has these dependencies in it’s buildscript:

buildscript {
    repositories {
        jcenter()
        maven {
            url "https://plugins.gradle.org/m2/"
        }
        mavenCentral name: "nexus", artifactUrls: ["http://mavencentral.it.whatever.com:8084/nexus/content/groups/whatever-public-group"]
        mavenLocal()
    }
    
    dependencies {
        classpath 'org.hidetake:gradle-ssh-plugin:2.1.0'
        classpath 'com.netflix.nebula:gradle-ospackage-plugin:4.3.0'
    }
}

This works just fine for me because com.whatever.subgroup:rpm_gradle:1.1.1 is in mavenlocal. It is also in http://mavencentral.it.whatever.com:8084/nexus/content/repositories/whatever-repository-releases. However, I find that if I delete the copy in mavenlocal, builds fail when they apply a plugin from rpm_gradle. They find ‘com.whatever.subgroup:rpm_gradle:1.1.1’ at http://mavencentral.it.whatever.com:8084/nexus/content/repositories/whatever-repository-releases but they fail to find com.netflix.nebula:gradle-ospackage-plugin:4.3.0. Even though this can be found at https://plugins.gradle.org/m2/, it seems that gradle doesn’t look there for plugin dependencies of the plugin, but only in the same repository in which it found the plugin.

http://mavencentral.it.whatever.com:8084/nexus/content/repositories/whatever-repository-releases is a mirror of Maven Central, which does not include the gradle plugins and is therefore not in the company mirror. But ‘com.whatever.subgroup:rpm_gradle:1.1.1’ is an internal plugin which will never be published in a public repository. The logical and the only possible place to publish it is in the corporate repository.

How can I configure this so that gradle will pull in rpm_gradle from the corporate repository and its dependency gradle-os-package plugin from https://plugins.gradle.org/m2/? Something is wrong here.


(James Justinic) #2

I can’t quite tell if this is really part of the problem or something was just lost in translation between the actual projects and the example posted, but Gradle doesn’t look for “plugin dependencies of the plugin” at all.

Your top level project is only going to resolve the actual dependencies of the rpm_gradle plugin project and add those to the buildscript classpath. The code shown only declares that you need the org.hidetake:gradle-ssh-plugin and com.netflix.nebula:gradle-ospackage-plugin packages on the classpath when building the rpm_gradle plugin, not when you’re using your plugin.


(Steve Cohen) #3

Thanks for responding! I hurriedly put this post together as I was getting ready to head home yesterday, and it’s too encrusted with details, but thanks for wading through them and basically understanding the situation.

So let’s simplify it.

Gradle Project A applies Plugin B (and has a dependency on it in its buildscript)
Plugin B applies and depends on Plugin C.

Plugin B and Plugin C do not and cannot live in the same repository, other than mavenLocal(), should they both live there. Plugin B lives in Repo B, the corporate Maven repository. Plugin C lives in https://plugins.gradle.org/m2/.

My experimentation shows me that unless Plugin B lives in mavenLocal(), Project A cannot be evaluated. And the root cause of the error is

Caused by: java.lang.ClassNotFoundException: com.netflix.gradle.plugins.rpm.Rpm

(i.e. plugin C)

In my original post I didn’t explain everything I have done. In Plugin B, we have both a buildscript “classpath” dependency and a normal “compile” dependency on plugin C. Both the buildscript and top level repositories collections include https://plugins.gradle.org/m2/.

The only way I find that I can build project A is if mavenLocal includes both plugin B and plugin C. It still seems to me as though the dependent plugin (plugin C) cannot be found unless it’s in the same repository as plugin B.

To put it even more succinctly, the problem seems to arise with

  1. a custom plugin that depends on another plugin
  2. which does not live in the same repository.

I find no mention of this scenario in the user’s guide, and although I have been using this plugin for over a year it only works with the “hack” of mavenLocal(). Which means, basically, that every developer who wants to work locally with this plugin must build it himself and publish it to his local maven repo.

How to remedy this?


(Steve Cohen) #4

Found a remedy. According to the User’s Guide, the behavior I identified as problematic is by design:

You may configure any number of repositories, each of which is treated
independently by Gradle. If Gradle finds a module descriptor in a
particular repository, it will attempt to download all of the artifacts
for that module from the same repository.
Although module meta-data and module artifacts must be located in the
same repository, it is possible to compose a single repository of
multiple URLs, giving multiple locations to search for meta-data files and jar
files.

Since projects using my plugin depend on three URLs, all three must be composed into a single “repository” for gradle purposes.

Therefore, the works:

In each project that depends on the dependent custom plugin:

buildscript {
    repositories {
        maven {
            url "http://mavencentral.it.whatever.com:8084/nexus/content/groups/whatever-public-group"
            artifactUrls "http://mavencentral.it.whatever.com:8084/nexus/content/repositories/whatever-repository-releases"
            artifactUrls "https://plugins.gradle.org/m2/"
        }
        mavenLocal()
     }
      dependencies {
        classpath 'com.whatever.subgroup:rpm_gradle:1.1.1'
     }
}

Where:
The main url is the corporate mirror of Maven Central.
The first artifact Url is the company repo for locally built artifacts
The second artifact Url is the official Gradle Plugin site.


(Stefan Oehme) #5

I think you misunderstood that part of the documentation. It just talks about the metadata and artifacts having to come from the same repo. That’s completely unrelated to transitive dependencies, which can of course come from a different repository.

So there’s no need to use artifactUrls, you can just define 3 maven { url =...} repositories. And I’d really recommend having one repository that mirrors all of those, so you don’t have to copy paste the same URLs in every one of your build scripts.

I just saw that this is what you tried initially. This should work. Can you create a self-contained example using a dummy plugin and a file-based Maven repository?


(Stefan Oehme) #6

Also if the dependency couldn’t be found you’d get a dependency resolution error. Getting a ClassNotFound looks much more like you are either missing the dependency declaration or the jar you got is corrupt for whatever reason.


(Steve Cohen) #7

Please explain how to set up a file-based maven repo?


(Stefan Oehme) #8

Using the maven or maven-publish plugin as usual, just using a local file instead of a remote url.


(Steve Cohen) #9

OK, I set up a simple project and have found that the repositories can either be single-url repos or combined and it works.

In my real-world situation, I also find that 3 single-url repos or 1 repo with 3 urls work the same. What seems to make a difference is that my initial configuration (see above) used mavenCentral{} rather than maven{}.

Unless there is some reason why the single repo with artifactUrls is bad practice, I think I prefer that way because it is a single block.

I have already explored whether our corporate repo folks could add https://plugins.gradle.org/m2/ but that takes me onto a bureaucratic playing field I’d just as soon avoid. So living with the 3-url repo sounds like my best option at this point.

Thanks.


(Steve Cohen) #10

There is a reason why it is bad practice. If I want to publish back to the corporate repo, I need to apply credentials. If I tell gradle that https://plugins.gradle.org/m2/ is part of my nexus repository, it will try to apply those credentials there. Thus it works best to have my corporate nexus as one repository and the gradle plugin rep as two maven repositories.


Gradle Plugin Repositories and Maven Central