Gradle compile task failed due to weird dependency looking up

GloryAndDream · March 9, 2023, 12:45pm

I configured gradle build task in Unbuntu but got strange errors, it went to look up the version of custom dependency which already exists in nexus, but still said can’t found it , why is that? And when I run gradle build on windows or intelij on windows , it is normal

enviroment:
Ubuntu 22.10 + gradle 8.0.2 + JDK 19
win11 + gradle 8.0.2 + JDK 18

project structrue:

machine-service-project
- machine-service-api
build.gradle
- machine- service-app
build.gradle
build.gradle

I go to subproject machine- service-app, and run command “gradle clean build”

machine-service-project/build.gradle

plugins {
    id 'java-platform'
    id 'maven-publish'
    //id 'io.spring.dependency-management' version '1.1.0'
}

javaPlatform {
    allowDependencies()
}
ext {
    nexus_url = System.getenv("GRADLE_ENV") == 'home' ? 'xxx.com' : '192.168.2.84'
    println('current nexus url is ' + nexus_url)
    commonLibsVersion = "0.0.1-SNAPSHOT"
}



allprojects {
    group = 'com.xxx'
    version = '0.0.1-SNAPSHOT'

    repositories {
        def urlStr = "http://${nexus_url}:8081/repository/maven-public/"
        mavenLocal()
        maven {
            url = uri(urlStr)
            allowInsecureProtocol = true
        }

        maven {
            url = uri('https://maven.aliyun.com/repository/public/')
        }
    }

    tasks.withType(JavaCompile) {
        options.encoding = 'UTF-8'
    }

    tasks.withType(Javadoc) {
        options.encoding = 'UTF-8'
    }

    repositories {
        def releasesRepoUrl = "http://${nexus_url}:8081/repository/maven-releases/"
        def snapshotsRepoUrl = "http://${nexus_url}:8081/repository/maven-snapshots/"
        def dist_url = version.endsWith('SNAPSHOT') ? snapshotsRepoUrl : releasesRepoUrl
        maven {
            url dist_url
            credentials {
                username = aa
                password = ***
            }
            authentication {
                basic(BasicAuthentication)
            }
            allowInsecureProtocol = true
        }
    }

}

subprojects {
    apply plugin: 'java-library'
    dependencies {
        api(platform("com.laiguaba:cloud-common-libs:${commonLibsVersion}"))
        api 'org.springframework.boot:spring-boot-starter'
        api 'org.springframework.boot:spring-boot-starter-web'

    }
}


publishing {
    publications {
        maven(MavenPublication) {
            from(components.javaPlatform)
        }
    }

}

machine-service-app/build.gradle:

dependencies {
    api "com.laiguaba:cloud-common-libs:${commonLibsVersion}"
    api 'com.alibaba.cloud:aliyun-oss-spring-boot-starter:'
    api 'com.alibaba.cloud:aliyun-rds-spring-boot-starter'
    api 'com.alibaba.cloud:spring-cloud-starter-stream-rocketmq'
    testImplementation 'org.springframework.boot:spring-boot-starter-test'
}

GloryAndDream · March 9, 2023, 12:47pm

The dependency in local nexus:

Vampire · March 9, 2023, 2:39pm

The error says you are requesting the dependency as “platform” or “bom” variant, but it is just a normal dependency.
This line: api(platform("com.laiguaba:cloud-common-libs:${commonLibsVersion}"))

GloryAndDream · March 9, 2023, 2:51pm

but it doesn’t explain why on windows it works fine?

Vampire · March 9, 2023, 2:56pm

I just explained what the error says.
My crystal ball is at the repair shop, I cannot guess what is different on your systems unfortunately.
Maybe on the Windows machine you have GRADLE_ENV set to home and on the Ubuntu machine not or the other way around and on one of the Nexuses it is a platform and on the other not.

Or it might be caused by you having the broken-by-design mavenLocal() in there and as first repo and without content filter specification, which per definition makes your builds flaky and slow. So maybe you have different content in your maven local on both machines that make it work on one but break on the other.

Vampire · March 9, 2023, 2:57pm

Can you maybe provide a build scan of both runs?

GloryAndDream · March 9, 2023, 3:31pm

the build on Ubuntu

bash /opt/gradle-8.0.2/bin/gradle build --scan

> Configure project :
current nexus url is 192.168.2.84

> Configure project :machine-service-app
##teamcity[setParameter name='project.version' value='0.0.1-SNAPSHOT']

> Task :machine-service-app:compileJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':machine-service-app:compileJava'.
> Could not resolve all files for configuration ':machine-service-app:compileClasspath'.
   > Could not resolve com.laiguaba:cloud-common-libs:0.0.1-SNAPSHOT.
     Required by:
         project :machine-service-app
      > No matching variant of com.laiguaba:cloud-common-libs:0.0.1-SNAPSHOT:20230307.162106-5 was found. The consumer was configured to find a platform for use during compile-time, compatible with Java 17, preferably in the form of class files, preferably optimized for standard JVMs, and its dependencies declared externally but:
          - Variant 'apiElements' capability com.laiguaba:cloud-common-libs:0.0.1-SNAPSHOT declares a component for use during compile-time, compatible with Java 17, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a library and the consumer needed a platform
              - Other compatible attribute:
                  - Doesn't say anything about its target Java environment (preferred optimized for standard JVMs)
          - Variant 'runtimeElements' capability com.laiguaba:cloud-common-libs:0.0.1-SNAPSHOT declares a component for use during runtime, compatible with Java 17, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a library and the consumer needed a platform
              - Other compatible attribute:
                  - Doesn't say anything about its target Java environment (preferred optimized for standard JVMs)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.

the build on windows:

 gradle build --scan

> Configure project :
current nexus url is hz.laiguaba.com

> Configure project :machine-service-app
##teamcity[setParameter name='project.version' value='0.0.1-SNAPSHOT']

BUILD SUCCESSFUL in 5s
5 actionable tasks: 5 up-to-date

Vampire · March 9, 2023, 6:22pm

You did run with --scan but you didn’t provide the links to the build scan

GloryAndDream · March 10, 2023, 7:21am

ubuntu build: Build Scan™ for ‘machine-service-project’ today at 15:12:26 CST | Gradle Cloud Services

windows build: Build Scan™ for ‘machine-service-project’ just now | Gradle Cloud Services

Vampire · March 10, 2023, 5:14pm

Yep, as I assumed.
In the build where it is working, it is coming from Maven local:

GloryAndDream · March 11, 2023, 2:28am

OK, but why is that the gradle searched this platform dependency in maven local rather than maven remote repo which is defined already? and it seems for other api dependency , gradle will search in remote repo

Vampire · March 11, 2023, 2:33am

Gradle searches the repos in the order you defined them.
And as I mentioned before, you have defined mavenLocal() as the first repo and without any content filter.
That means every dependency is first searched there and only if not available, in the next and next and next repository.
Which as mentioned, makes you builds slow (pure usage of mavenLocal(), not the amount of repositories), and unreliable and flaky as the Maven Local repository as designed and used by Maven is broken by design as mentioned and should be avoided wherever possible.

GloryAndDream · March 11, 2023, 2:48am

GloryAndDream:

    repositories {
        def urlStr = "http://${nexus_url}:8081/repository/maven-public/"
        mavenLocal()
        maven {
            url = uri(urlStr)
            allowInsecureProtocol = true
        }

        maven {
            url = uri('https://maven.aliyun.com/repository/public/')
        }
    }

as can be seen in the build.gradle, I already defined 3 maven repos, so if gradle can’t find the dependency, shouldn’t it go to find next maven repo?

also I tried to comment mavenLocal() line,

       // mavenLocal()

but still failed, seems it keeps looking up in mavenLocal(), could it be a bug for “java-platform” plugin?

Vampire · March 11, 2023, 3:52am

It looks in the other repos if it does not find it there.
But you probably got me wrong.
Try on the system where it is working to comment mavenLocal() and it will most probably fail the same way.
Most probably because on that system you published a version of that dependency to Maven Local that is a platform, while on the remote repos it is not, as explained previously.

GloryAndDream · March 11, 2023, 4:27am

Yes, you are right, I comment mavenLocal on windows, then it build failed

below is build.gradle for that dependency com.laiguaba:cloud-common-libs:0.0.1-SNAPSHOT

plugins {
    id 'java-library'
    id 'maven-publish'
}

ext {

    nexus_url = '192.168.2.84'
    println('current nexus url is ' + nexus_url)
}

repositories {
    def urlStr = "http://${nexus_url}:8081/repository/maven-public/"
    mavenLocal()
    maven {
         url = uri(urlStr)
         allowInsecureProtocol = true
    }

    maven {
        url = uri('https://maven.aliyun.com/repository/public/')
    }
}

group = 'com.laiguaba'
version = '0.0.1-SNAPSHOT'
java.sourceCompatibility = JavaVersion.VERSION_17

dependencies {
    api platform('org.springframework.boot:spring-boot-dependencies:3.0.2')
   api platform('org.springframework.cloud:spring-cloud-dependencies:2022.0.1')
    api platform('com.alibaba.cloud:spring-cloud-alibaba-dependencies:2022.0.0.0-RC1')
    api platform('com.alibaba.cloud:aliyun-spring-boot-dependencies:1.0.0')**
    // springboot \ spring cloud
    api 'org.springframework.boot:spring-boot-starter'
    api 'org.springframework.cloud:spring-cloud-starter-bootstrap:4.0.0'
    api 'org.springframework.boot:spring-boot-starter-webflux'
    api 'org.springframework.boot:spring-boot-starter-data-redis'
    compileOnly 'org.projectlombok:lombok:1.18.24'
    annotationProcessor 'org.projectlombok:lombok:1.18.24'
    api 'org.projectlombok:lombok-mapstruct-binding:0.2.0'
    // DB: mysql \ redis
    api 'com.mysql:mysql-connector-j:8.0.31'
    api 'com.baomidou:mybatis-plus-boot-starter:3.5.3.1'
    api 'com.baomidou:dynamic-datasource-spring-boot-starter:3.6.1'
    api 'io.lettuce:lettuce-core:6.2.3.RELEASE'
    api 'org.apache.commons:commons-pool2:2.11.1'
    api 'com.alibaba:druid-spring-boot-starter:1.2.16'

    api 'org.springframework.boot:spring-boot-starter-actuator:3.0.2'
    api 'org.springframework.boot:spring-boot-actuator:3.0.2'
    api 'org.springframework.boot:spring-boot-actuator-autoconfigure:3.0.2'
    api 'org.springframework.boot:spring-boot-configuration-processor:3.0.2'
    api 'org.springframework.cloud:spring-cloud-starter-kubernetes-client-all:3.0.1'

    api 'com.aliyun:aliyun-java-sdk-core:4.6.3'
    api 'com.aliyun:aliyun-java-sdk-dyplsapi:1.3.3'
    api 'com.aliyun.oss:aliyun-sdk-oss:3.15.1'
    api 'com.alipay.sdk:alipay-sdk-java:4.34.0.ALL'

    api 'cn.hutool:hutool-all:5.8.12'
    api 'io.swagger:swagger-annotations:1.5.22'
}

publishing {
    publications {
        maven(MavenPublication) {
            from(components.java)
        }
    }
    repositories {
        def releasesRepoUrl = "http://${nexus_url}:8081/repository/maven-releases/"
        def snapshotsRepoUrl = "http://${nexus_url}:8081/repository/maven-snapshots/"
        def dist_url = version.endsWith('SNAPSHOT') ? snapshotsRepoUrl : releasesRepoUrl
        maven {
            url dist_url
            credentials {
                username =xxx
                password = ***
            }
            authentication {
                basic(BasicAuthentication)
            }
            allowInsecureProtocol = true
        }
    }
}

tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}

tasks.withType(Javadoc) {
    options.encoding = 'UTF-8'
}

I put some common methods and library to this project, and I want other projects to reuse this common dependency, so I have to declare plugin type as “java-library”, but also I want to control the dependencies version only in this common project ,
e.g. for org.springframework.cloud:spring-cloud-dependencies:2022.0.1 etc,
just like the way of maven BOM

so in order to achieve my goal, how can I fix this issue and do it in proper way?

GloryAndDream · March 11, 2023, 3:09pm

Yes， I did and the build failed on windows

below is build.gradle of com.laiguaba:cloud-common-libs:0.0.1-SNAPSHOT

plugins {
    id 'java-library'
    id 'maven-publish'
}


repositories {

    mavenLocal()
    maven {
         url = uri(urlStr)
         allowInsecureProtocol = true
    }

}

group = 'com.laiguaba'
version = '0.0.1-SNAPSHOT'
java.sourceCompatibility = JavaVersion.VERSION_17

dependencies {
    api platform('org.springframework.boot:spring-boot-dependencies:3.0.2')
    api platform('org.springframework.cloud:spring-cloud-dependencies:2022.0.1')
    api platform('com.alibaba.cloud:spring-cloud-alibaba-dependencies:2022.0.0.0-RC1')
    api platform('com.alibaba.cloud:aliyun-spring-boot-dependencies:1.0.0')

    api 'org.springframework.boot:spring-boot-starter'
    api 'org.springframework.cloud:spring-cloud-starter-bootstrap:4.0.0'
    api 'org.springframework.boot:spring-boot-starter-webflux'
    api 'org.springframework.boot:spring-boot-starter-data-redis'
    compileOnly 'org.projectlombok:lombok:1.18.24'
    annotationProcessor 'org.projectlombok:lombok:1.18.24'
    api 'org.projectlombok:lombok-mapstruct-binding:0.2.0'

    api 'com.mysql:mysql-connector-j:8.0.31'
    api 'com.baomidou:mybatis-plus-boot-starter:3.5.3.1'
    api 'com.baomidou:dynamic-datasource-spring-boot-starter:3.6.1'
    api 'io.lettuce:lettuce-core:6.2.3.RELEASE'
    api 'org.apache.commons:commons-pool2:2.11.1'
    api 'com.alibaba:druid-spring-boot-starter:1.2.16'

    api 'org.springframework.boot:spring-boot-starter-actuator:3.0.2'
    api 'org.springframework.boot:spring-boot-actuator:3.0.2'
    api 'org.springframework.boot:spring-boot-actuator-autoconfigure:3.0.2'
    api 'org.springframework.boot:spring-boot-configuration-processor:3.0.2'
    api 'org.springframework.cloud:spring-cloud-starter-kubernetes-client-all:3.0.1'
   
    api 'com.aliyun:aliyun-java-sdk-core:4.6.3'
    api 'com.aliyun:aliyun-java-sdk-dyplsapi:1.3.3'
    api 'com.aliyun.oss:aliyun-sdk-oss:3.15.1'
    api 'com.alipay.sdk:alipay-sdk-java:4.34.0.ALL'

    api 'cn.hutool:hutool-all:5.8.12'
    api 'io.swagger:swagger-annotations:1.5.22'
}

publishing {
    publications {
        maven(MavenPublication) {
            from(components.java)
        }
    }
    repositories {

        maven {
            url dist_url
            credentials {
                username = xxx
                password = ***
            }
            authentication {
                basic(BasicAuthentication)
            }
            allowInsecureProtocol = true
        }
    }
}

tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}

tasks.withType(Javadoc) {
    options.encoding = 'UTF-8'
}

I have to define plugin ‘java-library’, because there are common methods in this project, and I also want to control the dependencies version in this project , like maven BOM,
e.g. define version of ‘org.springframework.cloud:spring-cloud-dependencies:2022.0.1’

for other project which use this cloud-common-libs, just need to define dependencies in the BOM file without sepcify version

so in order to achieve the goal, how can I do it in proper way?

Vampire · March 11, 2023, 5:01pm

Well, you only define that this project is a Java library.
Why do you expect you are able to consume it as platform? o_O
Maybe you can apply both, the java-platform and the java-library plugin?
I don’t know, never did that.

GloryAndDream · March 11, 2023, 5:18pm

Because I need other projects to only define some dependencies from BOM like org.springframework.boot:spring-boot-dependencies:3.0.2

e.g.
spring-boot-starter
spring-boot-web

but I don’t want other projects to declare verion numbers of above…

by the way java-platform and the java-library plugin can’t be defined both

Vampire · March 11, 2023, 5:31pm

Yeah, well, as the docs say.
A project is either a platform, or produces binaries, but not both.