Stop downloading from repository repo.maven.apache.org

Help/Discuss
1

My Jenkins build runs behind fire wall and doesn’t have access to external repository like repo.maven.apache.org or plugins.gradle.org. These external repositories are not included in my build.gradle as well. But during the build, the gradle build tries to download from these repositories although the POM files are already in the gradle cache. This behaviour slows the build down since it involves timeout and retries

2022-10-13T12:47:20.805+0000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Resolve dependencies of detachedConfiguration2' started
2022-10-13T12:47:20.808+0000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Download https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom' started
2022-10-13T12:47:20.810+0000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Download https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom'
2022-10-13T12:47:21.816+0000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Download https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom' started
2022-10-13T12:47:21.818+0000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Download https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom'

2022-10-13T12:47:20.807+0000 [DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver] Loading file:/root/.m2/repository/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom
2022-10-13T12:47:20.807+0000 [DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver] No meta-data file or artifact found for module 'io.quarkus:io.quarkus.gradle.plugin:2.13.0.Final' in repository 'MavenLocal'.
2022-10-13T12:47:20.807+0000 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.modulecache.PersistentModuleMetadataCache] Recording absence of module descriptor in cache: io.quarkus:io.quarkus.gradle.plugin:2.13.0.Final [changing = false]

2022-10-13T12:47:20.807+0000 [DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver] Loading https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom
2022-10-13T12:47:20.807+0000 [DEBUG] [org.gradle.internal.resource.transfer.DefaultCacheAwareExternalResourceAccessor] Constructing external resource: https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom
2022-10-13T12:47:20.808+0000 [DEBUG] [org.gradle.internal.resource.transport.http.HttpResourceAccessor] Constructing external resource: https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom
2022-10-13T12:47:20.808+0000 [DEBUG] [org.gradle.internal.resource.transport.http.HttpClientHelper] Performing HTTP GET: https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom

2022-10-13T12:47:20.810+0000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Download https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom' completed
2022-10-13T12:47:20.810+0000 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess] Error while accessing remote repository MavenRepo. Waiting 1000ms before next retry. 2 retries left
org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve io.quarkus:io.quarkus.gradle.plugin:2.13.0.Final.
Caused by: org.gradle.api.resources.ResourceException: Could not get resource 'https://repo.maven.apache.org/maven2/io/quarkus/io.quarkus.gradle.plugin/2.13.0.Final/io.quarkus.gradle.plugin-2.13.0.Final.pom'.

My first question is why it still tried to download from repositories since the POM already in local cache. When the build runs on my local machine, it simply uses the cached POM.
I also tried to turn off the external repositories download by add the following filter.

        maven {
		  url "https://repo.maven.apache.org/maven2/"
          content {
             includeGroup "nothing.*"
          }
        }

But it still tries to download it from external repositories.
I’m running Gradle 7.2 https://services.gradle.org/distributions/gradle-7.2-bin.zip

2

Hard to guess without seeing your build scripts.
It could be because of dynamic version, or because of changing versions, or because of explicitly requesting it, or because of the POM or Ivy file referencing a GMM file that was not published alongside, … many possible reasons.

That it looks in Maven Central even with you not having it configured could mean that you use a misbehaving plugin that adds the repository itself.
It technically can do that, but it is very bad practice as it for example makes problems with company guidelines mandating for only using an internal server or air-gapped machines.

You can define your repositories centrally in your settings file and there also enforce that only those repositories are used and fail the build if a plugin or build script tries to add other repositories.

To force Gradle to not look for dynamic version or changing version changes online, you can use the --offline switch or according property which will force it to use its cached value if one is available or otherwise fail the build.

3

Thank you, Vampire, for looking into the problem. I attached my build scripts of the main project and the subproject.

buildscript {
    ext.taasArtifactoryUsername =  project.hasProperty('taasArtifactoryUsername') ? project.getProperty('taasArtifactoryUsername') : System.getenv('USERNAME')
	ext.taasArtifactoryPassword = project.hasProperty('taasArtifactoryPassword') ? project.getProperty('taasArtifactoryPassword') : System.getenv('PASSWORD')
    
	repositories {		
		maven {
			url "https://artifactory.commops.truvenhealth.com/artifactory/wh-imaging-external-maven-virtual"
			credentials {
				username = "${taasArtifactoryUsername}"
				password = "${taasArtifactoryPassword}"
			}
		}
		
		maven {
			url "https://plugins.gradle.org/m2/"
			
		}
	}
	dependencies {
		classpath "org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.8"
		classpath "org.jfrog.buildinfo:build-info-extractor-gradle:4.18.0"
		
	}
	

	}



plugins {
	id 'java'
	id "org.sonarqube" version "3.0"
}

allprojects {
	group = 'com.merative.aio'
	version = '1.0.0-snapshot'
}

subprojects {
	apply plugin: 'java'
	
	repositories {
		
		
		maven {
			url "https://artifactory.commops.truvenhealth.com/artifactory/wh-imaging-external-maven-virtual"
			metadataSources {
				mavenPom()
				artifact()
			}
			credentials {
				username = "${taasArtifactoryUsername}"
				password = "${taasArtifactoryPassword}"
			}
		}
		
		maven {
			url "https://www.dcm4che.org/maven2/"
			
		}
		
		maven {
			url "https://plugins.gradle.org/m2/"
			
		}
		
		maven {
			url "https://artifactory.commops.truvenhealth.com/artifactory/wh-img-sln-maven-local/"
			metadataSources {
				mavenPom()
				artifact()
			}
			credentials {
				username = "${taasArtifactoryUsername}"
				password = "${taasArtifactoryPassword}"
			}
		}
		
        maven {
		  url "https://repo.maven.apache.org/maven2/"
          content {
            // Don't get anything from mavenCentral since Jenkins build server can't reach it
            includeGroup "nothing.*"
          }
        }
		 
		
	}

	compileJava {
		options.encoding = 'UTF-8'
		options.compilerArgs << '-parameters'
	}

	compileTestJava {
		options.encoding = 'UTF-8'
	}

	java {
		sourceCompatibility = JavaVersion.VERSION_11
		targetCompatibility = JavaVersion.VERSION_11
	}	


	test {
		minHeapSize = '64M'
		maxHeapSize = '1G'
		useJUnitPlatform()
		systemProperty "java.util.logging.manager", "org.jboss.logmanager.LogManager"
		testLogging {
			events "failed"
			exceptionFormat "full"
		}

	}
	
	sonarqube {
    	properties {      
        	property "sonar.coverage.jacoco.xmlReportPaths", "${buildDir}/jacoco-report/jacoco.xml"
    	}

	}
}

Subproject:


plugins {
    id 'java'
    id 'io.quarkus'
   
}


ext{
withoutLOGTransitives = {
        exclude group:"ch.qos.logback", module:"logback-core"
        exclude group:"ch.qos.logback", module:"logback-classic"
        exclude group: 'org.apache.logging.log4j', module: 'log4j-api'

    }}
dependencies {

implementation project(':cloud-events-common')
	
    implementation enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
    
    implementation 'io.quarkus:quarkus-container-image-docker'
    implementation 'io.quarkus:quarkus-arc'
    implementation 'io.quarkus:quarkus-smallrye-context-propagation'
    implementation 'io.quarkus:quarkus-smallrye-health'
    implementation 'io.quarkus:quarkus-smallrye-fault-tolerance'
    implementation 'io.quarkus:quarkus-undertow'
    implementation 'org.eclipse.microprofile.context-propagation:microprofile-context-propagation-api:1.2'  
    implementation "org.bitbucket.b_c:jose4j:0.7.9"
   // unit testing
       
    testImplementation group: 'io.quarkus', name: 'quarkus-junit5'
    testImplementation 'org.assertj:assertj-core:3.9.1'
    testImplementation group: 'berlin.yuna', name: 'nats-server', version: '2.6.9'
    testImplementation group: 'io.rest-assured', name: 'rest-assured', version: '4.4.0'

}

The internal repository is actually defined in settings.gradle as well.
After adding --offline option, the build failed with following error.

2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Plugin [id: 'org.sonarqube', version: '3.0'] was not found in any of the following sources:
2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] 
2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] - Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] - Plugin Repositories (could not resolve plugin artifact 'org.sonarqube:org.sonarqube.gradle.plugin:3.0')
2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]   Searched in the following repositories:
2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]     MavenLocal(file:/root/.m2/repository)
2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]     MavenRepo
2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]     Gradle Central Plugin Repository
2022-10-25T16:33:57.271+0000 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]     maven(https://artifactory.commops.truvenhealth.com/artifactory/wh-imaging-external-maven-virtual)
4

And how does your settings script look like?
Seems to me as if you haven Maven Central as plugin repo.
(as well as Maven Local which should imho be never used for anything and the Gradle plugin repository).

The error you get with --offline should mean that you do not have that plugin in the local Gradle cache and thus it cannot continue.

5

You are right that the plugin is not in the cache of the build machine. So it sounds the --offline option can’t be used, right?
Here is my settings.

pluginManagement {
  ext.taasArtifactoryUsername =  System.getenv('USERNAME')
	ext.taasArtifactoryPassword = System.getenv('PASSWORD')
 
    repositories {
        mavenLocal()
        mavenCentral()
        gradlePluginPortal()
        maven {
			url "https://artifactory.commops.truvenhealth.com/artifactory/wh-imaging-external-maven-virtual"
			credentials {
				username = "${taasArtifactoryUsername}"
				password = "${taasArtifactoryPassword}"
			}
		}
    }
    plugins {
      id 'io.quarkus' version "${quarkusPluginVersion}"
    }
}

rootProject.name = 'aio-java'
include 'cloud-events-common'
6

I don’t get you.
You initially said the stuff is in the cache already and you want to prevent the build to try contacting the repos which is exactly what --offline does.
Now you say the stuff is not in the cache so you cannot use --offline.
But if it is not in the cache and you cannot access the repositories, how should it work then?

7

There are internal repositories like https://artifactory.commops.truvenhealth.com/artifactory.... The build is expected to pick up the dependencies from all defined repositories but not the ones which are brought in by the plugins.
When the build first starts, the gradle cache is empty. After the dependencies are collected from internal repositories and added to the local gradle cache, it looks gradle still goes out to the external repositories which are blocked by the firewall.

8

I still don’t really get what you are saying.
You explicitly define mavenCentral() and gradlePluginPortal() to be looked at before your internal repository and that is what Gradle does and what you say is expected.
So everything seems to be like expected.

9

Thank you so much, Vampire. I didn’t realize that these repositories are included in settings.gradle. Once they are removed, it works fine.