What is the correct URL to mirror plugins repository?

plugins
gradle-3552

(Matt Newman) #1

I’m working behind a corporate firewall, which doesn’t allow access to JARs hosted externally.

External Maven repositories such as JCentre are mirrored in an internal Nexus, so that we can access them internally.

Is there a Maven repository that can be used for mirroring plugins hosted on https://plugins.gradle.org/m2/?

This post on the old forum is kind of related, but doesn’t cover what Maven repository should be mirrored (or if there is one available).

Looking at what happens when I browse externally, I noticed

https://plugins.gradle.org/m2/gradle/plugin/com/gorylenko/gradle-git-properties/gradle-git-properties/1.4.7/gradle-git-properties-1.4.7.pom

redirects to

https://gradleware-plugins.s3.amazonaws.com/gradle.plugin.com.gorylenko.gradle-git-properties/gradle-git-properties/1.4.7/ae7c937023bd6eb15ba4bddf8c149b7d9c527dcef8bbbb20d829b3304312de5b/gradle-git-properties-1.4.7.pom

however, if you request anything that isn’t a full path to a file from https://plugins.gradle.org/m2/, you’re redirected to https://jcenter.bintray.com/. None of the plugins seem to exist where I’d expect them to in jCentre (e.g. there is nothing at https://jcenter.bintray.com/gradle/plugin/com/gorylenko/gradle-git-properties/gradle-git-properties/1.4.7/gradle-git-properties-1.4.7.pom).

Sorry if this post is a little rambling - I’m no Maven expert.


Can I use the new plugin DSL with a Nexus proxy repository?
(Tadhg Hamill) #2

I am also having this exact issue. This is currently preventing us from migrating to the new version of the Sonar Runner plugin as we have no way of mirroring the Gradle Plugins repo via our local Nexus server


(Tom Dunstan) #3

Hi guys

The plugin portal actually serves up an amalgam of three different things:

  • plugins published to the current portal, which are served up by S3 as you have observed
  • plugins published in the “old” way to Bintray and served up by JCenter
  • plugin dependencies which are served by JCenter

Due to the way that plugin resolution works for the plugins { } block in Gradle, Gradle expects plugins and their dependencies to be resolved from the same repo when using that construct. That’s why you see some requests redirecting to S3 and some to JCenter.

This does make automatic mirroring difficult as we can’t currently serve up a file list that spans all three sources of artifacts.

The best suggestion that I can offer right now is manually mirroring the jars / pom files you need for specific plugins that you use into a local repo.


(Erik Eloff) #4

I’m also behind a firewall that prevents Gradle from fetching artifacts from external repositories. It is rather cumbersome to manually mirror the needed plugins (and keeping them updated).

It is unfortunate that the “new and easy way” of plugins is harder to use than the old way, due to the firewall.
Are there any plans to make this easier?


(David Kowis) #5

I’m in the same boat. Having a legit maven repository that can be added to our Company’s artifactory pro as a proxy would be ideal. It makes it frustratingly difficult to get work done :frowning:


(René Groeschke) #6

you still can mirror https://plugins.gradle.org/m2/ in your artifactory at the moment right? We’re currently working on making the default repository for plugins configurable and you can expect something in one of the next gradle releases.


(David Kowis) #7

if I hit that url, it gives me a 404 error. Should that be the case?


(René Groeschke) #8

We have it on our long todo list to put a landing page there. please be patient. the url should work though in your artifactory/nexus.


(David Kowis) #9

I understand. Thanks for letting me know.


(David Kowis) #10

We configured our artifactory pro to point to https://plugins.gradle.org/m2 as a maven proxy repo, and it didn’t seem to work. :frowning:


(Trung Vo) #11

out CI server (bamboo) randomly fails because it couldn’t contact gradle plugins portal

Caused by:
org.gradle.internal.resource.transport.http.HttpRequestException: Could
not GET
https://plugins.gradle.org/api/gradle/2.13/plugin/use/com.dorongold.task-tree/1.2.1

Couple questions I have:

  1. is there any other repo that the gradle plugins are uploaded to?
  2. do you have any estimate when you guys can add the proxying abilities for https://plugins.gradle.org/m2?
  3. What do you mean by the “url should work though”, is there any other settings/url we can try out?
    In the worst case scenario, I might need to upload those plugins artifact to our local internal repository temporarily. Would you recommend that? is there any other options?

Cheers,
Tony


(Marcel Trautwein) #12

Found Plugin Repository Specification which seems to be an incubating way to solve this.

see also, that pluginRepositories.gradlePluginsPortal() should be configurable by systemProp.PluginResolutionServiceResolver.repo.override
https://github.com/gradle/gradle/blob/master/subprojects/plugin-use/src/main/java/org/gradle/plugin/use/resolve/service/internal/PluginResolutionServiceResolver.java#L72

not yet found a valid setup for me, but I’m sure I’m on the right way :wink:

Update: what seems to be a bit strange to me (or I did anything wrong): pluginRepositories {} was accepted silently in gradle 2.10 but has no effect but was declined in gradle 2.14.1-rc-1 ?!?!


(Stephan Krull) #13

Any news on that issue? The alternative M2 Repo is not available. Configuring it in Nexus version 2.6 or 3.0 results in “Blocked” status.

The migration to Gradle here depends partly on the realization to proxy the Gradle Plugin portal. Are there any changes in Gradle 3.x?

Regards


(Tobias Kremer) #14

Please get the M2 Repo working again. Without it there is no way to get plugins behind a firewall.
If you cannot: Please provide another way to fetch the plugins from behind a firewall.

I do not think it is OK that this problem is now ignored for month. It makes using Gradle in an enterprise environment quite hard.


(Marco Vermeulen) #15

@krulls @Endron I’d like to better understand the problem that you are experiencing, as to my knowledge the M2 Repo has been working all along. It acts as a proxy to S3, or JCenter as @tomd described in an earlier post.

If you try hitting the URL directly on the /m2 resource as above, you will get a 404. However, if you try hitting it with a valid URL such as https://plugins.gradle.org/m2/org/apache/httpcomponents/httpcore/4.1, you will be redirected to an appropriate resource (in this case Bintray).

Could it be that the 404 on /m2 is causing Nexus to break?


(Tobias Kremer) #16

@vermeulen_mp I am sorry. Reading my post now I realize I was kind of more angry then was actually needed.
I was under the wrong assumption that the M2 repository just does not work but the link you provided clearly showes that it does.

To the details: We are trying to proxy the Maven repo using Artifactory. But resolution of new artifact versions is not working. When I click the “Test” button in my Artifactory repository configuration screen I get an 404 error showing up. I am not sure if this then causes Artifactory to mark the whole thing as offline. (This is not clear from the documentation https://www.jfrog.com/confluence/display/RTF/Remote+Repositories .)
What is clear that our configuration in Artifactory worked at some point. We have sonarqube plugin 1.2 in there so it looks to me it worked at some point. But when I tried to update to 2.0 the artifact could not be resolved.


(Marco Vermeulen) #17

@Endron no problem at all, I would love to help you get this sorted as soon as possible. I’m sure that many are struggling with this same issue.

My guess is that usually proxies like this are simply a transparent proxy (like an nginx instance) which would mean that it would respond with something like jcenter does when the root context is requested.

What makes our endpoint different is that it is an intelligent proxy. Although I would not be able to give a listing, I would be able to return a 200 status when you hit our /m2 endpoint.

I think we should do this as a first call, then re-evalute to see if this solves your problem. I’ll post here when this has been implemented.


(Tobias Kremer) #18

@vermeulen_mp thank you.


(Stephan Krull) #19

I agree to @vermeulen_mp that sending a valid return code could be the missing peace in this. I can check with a Nexus 2 and Nexus 3 instance when this has been realized.

@@vermeulen_mp: I am missing links to an issue in an issue tracker to follow the workflow. Don’t you use one? Or is it private? Is there any other way than waiting for a new reply to this post to track the progress?


(James Justinic) #20

Adding a page that returns a status code of 200 will alleviate much of the confusion that has been expressed on the forums about proxying the Gradle Plugins repository. Most users expect that they can just create a proxied repository with the default settings, see success messages, and be on their way.

Nexus has required non-default settings since the plugin portal was created in order for it to work. It doesn’t work unless you know to change the default settings (due to the 404).

Artifactory gives you the sense that it doesn’t work because hitting to test button shows an error (also due to the 404). I’ve not seen any Artifactory issue that wasn’t actually caused by some other underlying issue, but getting a good test result would help speed up investigation.