Dependencies('requires' in module-info.java) not resolved when using custom configurations in build.gradle

Help/Discuss
1

For the following piece of code, the “requires org.postgresql.jdbc;” is unresolved when trying to perform Gradle-sync/assemble/build and using custom configuration ‘driver’ and custom sourceSet ‘driver’.

build.gradle

plugins {
    id 'java-library'           //used to create java api and implementation separated libs
}

configurations{
    driver
}

dependencies {
    driver group: 'org.postgresql', name: 'postgresql', version: '42.6.0'
}

sourceSets{
    driver{
        java{ srcDirs = ['src'] }
    }
}
compileJava.dependsOn(compileDriverJava)

module-info.java

module database.driver {
    requires java.base;              // core dependencies(internal)
    requires java.sql;               // sql module (externalized) dependencies
    requires org.postgresql.jdbc;    // postgresql driver (internal) dependencies
}

while it is resolved and builds/assembles fine without issues when using the default ‘implementation’ configuration and default ‘main’ sourceSet.

build.gradle

plugins {
    id 'java-library'           //used to create java api and implementation separated libs
}

dependencies {
    implementation group: 'org.postgresql', name: 'postgresql', version: '42.6.0'
}

sourceSets{
    main{
        java{ srcDirs = ['src'] }
    }
}
compileJava.dependsOn(compileDriverJava)

This is from CHAT-GPT:
enter image description here

Why are the custom configuration dependencies not visible for module-info.java, is this due to the reason that sourceSet ‘main’ is not used?

NOTE: running ./gradlew dependencies when using custom configuration ‘driver’ for sourceSets and dependencies task, prints out that dependencies are resolved.

NOTE: compileJava itself doesn’t do anything nor has any sourceSets to work with and in-turn doesn’t need any dependencies. But it depends on custom configuration compilation ‘compileDriverJava’

NOTE: postgres manifest contains Automatic-module-name to denote it is an automatic module

Manifest-Version: 1.0
Automatic-Module-Name: org.postgresql.jdbc

output of ‘./gradlew dependencies’ when using custom configuration ‘driver’

> Task :dependencies

------------------------------------------------------------
Root project 'database.driver'
------------------------------------------------------------

annotationProcessor - Annotation processors and their dependencies for source set 'main'.
No dependencies

api - API dependencies for source set 'main'. (n)
No dependencies

apiElements - API elements for main. (n)
No dependencies

compileClasspath - Compile classpath for source set 'main'.
No dependencies

compileOnly - Compile only dependencies for source set 'main'. (n)
No dependencies

compileOnlyApi - Compile only API dependencies for source set 'main'. (n)
No dependencies

default - Configuration for default artifacts. (n)
No dependencies

driver
\--- org.postgresql:postgresql:42.6.0
     \--- org.checkerframework:checker-qual:3.31.0

driverAnnotationProcessor - Annotation processors and their dependencies for source set 'driver'.
No dependencies

driverCompileClasspath - Compile classpath for source set 'driver'.
No dependencies

driverCompileOnly - Compile only dependencies for source set 'driver'. (n)
No dependencies

driverImplementation - Implementation only dependencies for source set 'driver'. (n)
No dependencies

driverRuntimeClasspath - Runtime classpath of source set 'driver'.
No dependencies

driverRuntimeOnly - Runtime only dependencies for source set 'driver'. (n)
No dependencies

implementation - Implementation only dependencies for source set 'main'. (n)
No dependencies

mainSourceElements - List of source directories contained in the Main SourceSet. (n)
No dependencies

runtimeClasspath - Runtime classpath of source set 'main'.
No dependencies

runtimeElements - Elements of runtime for main. (n)
No dependencies

runtimeOnly - Runtime only dependencies for source set 'main'. (n)
No dependencies

testAnnotationProcessor - Annotation processors and their dependencies for source set 'test'.
No dependencies

testCompileClasspath - Compile classpath for source set 'test'.
No dependencies

testCompileOnly - Compile only dependencies for source set 'test'. (n)
No dependencies

testImplementation - Implementation only dependencies for source set 'test'. (n)
No dependencies

testResultsElementsForTest - Directory containing binary results of running tests for the test Test Suite's test target. (n)
No dependencies

testRuntimeClasspath - Runtime classpath of source set 'test'.
No dependencies

testRuntimeOnly - Runtime only dependencies for source set 'test'. (n)
No dependencies
2

Ok, let’s start with: Don’t ask ChatGPT for anything you don’t know already anyway.
As you can see, answers of ChatGPT are basically always for the wastebasket.
The text you screenshotted is total non-sense and full of false-facts.
ChatGPT cannot provide real answers, it can provide answers that look real and sound nice.
It imho is a fancy toy and nothing more.
Its data base is more than 2 years old, which is an eternity in technology time.
It invents things that it things might fit into the context that are total non-sense.
I’ve also seen it produce code calling APIs with methods that do not exist and also never existed in those APIs, wherever it got that idea from.
The only case where you imho can properly use it so far is, for something that you could do yourself as well but are too lazy (the good lazy, not the bad one), as you need to be able to see the non-sense it created and need to be able to either fix it or at least tell it what it needs to change.

Your problem is, that you create a custom configuration that you nowhere use.
By creating a source set, you automatically get the proper configurations you need for that source set also with the proper attributes set. So declare your dependency for driverImplementation and it should work properly.

Besides that compileJava.dependsOn(compileDriverJava) does not really make sense.
And even if it would be technically correct, any dependsOn that does not have a lifecycle task on the left-hand side is a code smell and most probably a symptom of doing something else wrongly. Because if you properly wire outputs to inputs, you get necessary task dependencies automatically.

3

ChatGPT answers are also banned from StackOverflow for good reasons: Temporary policy: ChatGPT is banned - Meta Stack Overflow

4

Thanks for your reply, I didn’t mean to emphasize that I was depending on ChatGPT for any facts or real work, as a software developer myself I have seen how much of a garbage it can produce.

The chat GPT snippet was to (sarcastically) show that it is completely useless and just a search engine that spits out mix and matched content from random stackoverflow answers or database.

BTW, I had already tried your suggestion of using “driverImplementation” instead of “driver” in the dependency block, But couldn’t get it to work.

I think I am missing something here, Didn’t get much time due to work and other activities. Will post in another reply what I had tried.

1 Like
5

The fun part with chatGPT is that it is so broken that the answers it spits out can be tuned so much into a completely different topic by asking it to correct itself each time as a feedback.

maybe it will improvise but surely it didn’t help even 0.01% in my case as I clearly know what it suggested was clearly not the solution.

I preferred an actual human to help out and that is the reason I had posted here in gradle forums😊

1 Like
6

What happened when you did?

7

okay found the issue, seems gradle parser cares about sequence of token declaration,

The issue was that sourceSets was declared after dependencies block and by the time the dependencies block was resolved, there was only one more configuration called driver available in addition to implicit configuration default

So I was able to use driver group: 'org.postgresql', name: 'postgresql', version: '42.6.0' and it worked because it associated the dependency to the custom configuration itself.

But it was failing to see that there was a custom sourceSet driver for which the dependencies were specified by driverImplementation group: 'org.postgresql', name: 'postgresql', version: '42.6.0' as @Vampire suggested and which I had tried already before posting here.

So moving the sourceSets declaration above dependencies and having driverImplementation instead of driver resolved the issue.

NOTE: I had renamed the custom sourceSet driver to foo to check if it has any interlink to custom configuration, and the answer is IT DID NOT.
so basically if I rename the statement in the dependencies block to fooImplementation group: 'org.postgresql', name: 'postgresql', version: '42.6.0' after adding sourceSets { foo }, then gradle is able to map the dependencies properly to the custom sourceSet foo.

I am unsure if the gradle user-manual contains these minute detail/clarifications precisely from a new user perspective.

NOTE: Though my issue with gradle is resolved, it is not resolved entirely since I am working on IntelliJ IDE and it still doesn’t see/recognize the dependencies in custom SourceSet and shows errors in module-info.java but gradle compilation works fine even in IDE.

The IDE can see the dependencies and NOT produce errors visually when the dependencies are setup for main sourceSet using implementation group: 'org.postgresql', name: 'postgresql', version: '42.6.0' in the dependencies block.

Another workaround is for implementation configuration to extend from this custom configuration driver using implementation.extendsFrom(driver), then just using driver group: 'org.postgresql', name: 'postgresql', version: '42.6.0' so that the dependency associates to custom configuration driver and hence implicitly associates to implementation as well.

NOTE: Doing this resolves the dependency in IntelliJ IDE and IDE red marks are removed.

Case-1:

// DOESN'T WORK:
/*
    Reasons:
        1. sourceSets block declared after dependency block which contains the dependency
        2. it will still fail because there is no implementation dependency for the custom sourceSet ie, missing `driverImplementation`
*/

configurations {
    driver
}

dependencies {
    driver 'org.postgresql:postgresql:42.6.0'
}

sourceSets{
    driver{
        java{
            srcDirs = ['src/main/java']
        }
    }
}

task compile(){                            //GRADLE-ERROR
    dependsOn compileJava                  
    dependsOn compileDriverJava            
}

Case-2:

// DOESN'T WORK:
/*
    Reasons:
        1. there is no implementation dependency for the custom sourceSet ie, missing `driverImplementation` because dependencies declaration are unique for custom configuration and custom sourceSets.
        2. both sourceSets 'main' and 'driver' (custom sourceSet), FAILS
*/

configurations {
    driver
}

sourceSets{
    driver{
        java{
            srcDirs = ['src/main/java']
        }
    }
}

dependencies {
    driver 'org.postgresql:postgresql:42.6.0'
}

task compile(){
    dependsOn compileJava                  //FAILS
    dependsOn compileDriverJava            //FAILS
}

Case-3:

PARTIALLY WORKS:
/*
    Reasons:
        1. there is implementation dependency for the custom sourceSet ie, `driverImplementation` contains required dependency.
    NOTE: IDE still doesn't recognize the dependency since it is not on the main sourceSet.
        2. hence compileJava for main sourceSet, FAILS.
        3. however compileDriverJava for custom sourceSet 'driver', PASSES
*/

configurations {
    driver
}

sourceSets{
    driver{
        java{
            srcDirs = ['src/main/java']
        }
    }
}

dependencies {
    driverImplementation 'org.postgresql:postgresql:42.6.0'
}

task compile(){
    dependsOn compileJava                  //FAILS
    dependsOn compileDriverJava            //PASSES
}

Case-4:

PARTIALLY WORKs:
/*
    Reasons:
        1. though there is NO implementation dependency for the custom sourceSet ie, `driverImplementation`, the main sourceSet indirectly extends the custom configuration's dependency for itself.
        2. hence they become available and visible to the main sourceSet and IDE is able to resolve them too.
        3. but due to REASON-1 the compilation of custom sourceSet fails ie, task compileDriverJava fails
       4. however compileJava from main sourceSet passes and generates class files successfully.
*/

configurations {
    driver
    implementation.extendsFrom(driver)
}

sourceSets{
    driver{
        java{
            srcDirs = ['src/main/java']
        }
    }
}

dependencies {
    driver 'org.postgresql:postgresql:42.6.0'
}

task compile(){
    dependsOn compileJava                  //PASSES
    dependsOn compileDriverJava            //FAILS
}

Case-5

FULLY WORKs:
/*
    Reasons:
        1. there is implementation dependency for the custom sourceSet ie, `driverImplementation`.
        2. due to REASON-1, compileDriverJava, PASSES
        3. the implementation configuration extends from custom configuration driver inheriting all its dependencies.
       4. due to REASON-3, compileJava, PASSES.
*/
configurations {
    driver
    implementation.extendsFrom(driver)
}

sourceSets{
    driver{
        java{
            srcDirs = ['src/main/java']
        }
    }
}

dependencies {
    driver 'org.postgresql:postgresql:42.6.0'
    driverImplementation 'org.postgresql:postgresql:42.6.0'
}

task compile(){
    dependsOn compileJava                  //PASSES
    dependsOn compileDriverJava            //PASSES
}
8

The issue was that sourceSets was declared after dependencies block and by the time the dependencies block was resolved, there was only one more configuration called driver available in addition to implicit configuration default

Ah, yes, of course.
That’s not a matter of parser.
The build scripts are scripts in a turing-complete language that are executed line by line, so if some block is not executed deferred somhow, it is executed right away.
sourceSets { driver } creates the source set and the according configurations.
Only after that you can use those configurations, for example to add dependencies to them.

I am unsure if the gradle user-manual contains these minute detail/clarifications precisely from a new user perspective.

Yes: Building Java & JVM projects :slight_smile:

it is not resolved entirely since I am working on IntelliJ IDE and it still doesn’t see/recognize the dependencies in custom SourceSet

Are you sure you synchronized the Gradle project after setting it up properly?
Should work just fine in IntelliJ.
Can you provide an MCVE where it does not?

Another workaround is for implementation configuration to extend from this custom configuration driver using implementation.extendsFrom(driver)

That would in your case practically be the same as just declaring the dependency on implementation itself, so not really what you want.

Regarding your cases:

Case 1: Of course fails, you just add the dependency to the not-used driver configuration that has no relation at all to the source set.

Case 2: Same as Case 1

Case 3: You can remove the useless driver configuration, besides that it is declared in a legacy way you don’t use it anymore at all. That compileJava still fails is clear if you poing both source sets to the same source files but only give one source set the needed dependency. But then, where it the point in having two source sets? That is the point that makes absolutely no sense here and causes you problems.

Case 4: Of course compileJava works as like described above, your extendsFrom in your case is exactly the same as if you would declare the dependency on implementation directly and you do not add the dependency to the driver source set. So just like Case 3, you point your source sets to the same sources but give only one the needed dependency and the other not, so of course the one you do not give it fails.

Case 5: Even if repeating myself, that setup makes absolutely no sense. You basically just compile the same sources twice and have a pointless additional configuration that has absolutely no use.

9

@Vampire Thank you for keeping up with me on this.

And Yes, I think I should’ve paid more attention to the custom sourceSets section in addition to custom configurations section.

of course gradle manual is user friendly and helped me many times now, but sometimes I do get lost in the sea of contents! :innocent:

Also thanks again for asserting the results of the cases I had mentioned, I just had put them there to confirm what happens, if some new person stumbles upon such issue and want a quick context around the issue.

Lastly, as you said, I just need to reconfirm once that the IDE issue is persistent since I am sure that I had performed gradle sync and task compileDriverJava was SUCCESSFUL too.

So let’s say IntelliJ LINTer cannot see dependencies that aren’t associated with main sourceSet but gradle is HAPPY.

NOTE: this wouldn’t be an issue on modules that doesn’t have a module-info.java both with java8 or java9+. IntelliJ suggests that I “add the dependency to the classpath” using the :bulb:icon.
When I click to accept the suggestion nothing happens, It still asks the same suggestion and there is no changes made to build.gradle.
I was expecting it to add an implementation clause to dependencies for postgres or at least concatenate the runtime classpaths of main and driver sourceSet in driver both of which are probably NOT REAL SOLUTION to the problem.

seems like a bug in IDE linter, I will try for the minimal reproducible example as you requested and will post here soon. :slightly_smiling_face:

10

Lastly, as you said, I just need to reconfirm once that the IDE issue is persistent since I am sure that I had performed gradle sync and task compileDriverJava was SUCCESSFUL too.

Well, as I said, none of your configurations, even the one that works with Gradle, makes too much sense.
So whether it works in IntelliJ or not, might not be too relevant, as the setup just makes no sense. But yeah, it should work indeed even if it does not make sense.

So let’s say IntelliJ LINTer cannot see dependencies that aren’t associated with main sourceSet but gradle is HAPPY .

No, let’s not say that.
IntelliJ has no evaluation, it just asks Gradle “give me information about this project” and then transforms it into a proper IntelliJ model. It does not interpret the build scripts itself, which also would be impossible and ridiculous as they are written in turing-complete languages.

IntelliJ suggests that I “add the dependency to the classpath” using the :bulb:icon.
When I click to accept the suggestion nothing happens, It still asks the same suggestion and there is no changes made to build.gradle.

Those quick-fixes do not edit build scripts, they would only add the dependency in the IntelliJ model and even if it would work, it would be void as soon as the Gradle project is resynced.

11

Thanks for getting back,

Yeah, those configs were just for common understanding on which configurations causes what effects (for quick glance).

I have found the issue with my config, on why IntelliJ still doesn’t see the dependencies though they are on the driver sourceSet’s runtime classpath (driverImplementation).

The issue is that I had unknowingly mapped ‘driver’ sourceSet onto ‘main’ sourceSet

    driver{
        java{
            srcDirs = ['src/main/java']
        }
    }

Though I haven’t declared main sourceSet explicitly, but it is implicitly available and is mapped onto src/main/java which is same as driver’s java sourceSet.

So if I move the sources from src/java/main into src/driver/java and use the following

    driver{
        java{
            srcDirs = ['src/driver/java']
        }
    }

Then IntelliJ is happy and is able to see/resolve the requires modules in module-info.java

I think the reason why it was unable to resolve is due to the fact that implicit sourceSet main didn’t have the required dependency in its runtime classpath (implementation).

Thoughts:

  • Is there a priority/order for sourceSets that intelliJ uses for Linting (so as to determine if two or more sourceSets point to same source path and if sourceSet which is missing the dependency gets priority such that IntelliJ can’t resolve dependencies but gradle is happy).

Note: main sourceSet after I had moved the sources from src/main/java to src/driver/java, doesn’t contain any java/module-info.java files. (Hence no more complaints from IntelliJ?)

12

The issue is that I had unknowingly mapped ‘driver’ sourceSet onto ‘main’ sourceSet

Yes, that’s exactly what I told you. :slight_smile:

So if I move the sources from src/java/main into src/driver/java and use the following

Just remove that part, it is unnecessary boilerplate. src/driver/java is the conventional default anyway.

I think the reason why it was unable to resolve is due to the fact that implicit sourceSet main didn’t have the required dependency in its runtime classpath (implementation).

Yes, that’s exactly what I told you. :slight_smile:

  • Is there a priority/order for sourceSets that intelliJ uses for Linting (so as to determine if two or more sourceSets point to same source path and if sourceSet which is missing the dependency gets priority such that IntelliJ can’t resolve dependencies but gradle is happy).

Gradle also was not happy unless you added the dependency to both source sets.
And at that point IntelliJ should also have been happy, even though the setup was totally senseless by then.
You shouldn’t expect IntelliJ to be happy about something Gradle is unhappy about.

13

But gradle was happy in my case since it didn’t throw errors when driver sourceSet was mapped onto main’s.

compileDriverJava task was passing as expected and compileJava was failing as expected too.

So I don’t see gradle unhappy (ie unresolved dependencies unlike intelliJ)

14

compileJava was failing

=> unhappy

15

Hmm, understandable, So basically IntelliJ wants all sourceSets to be resolvable at Gradle Sync phase.

Yeah and that is how it should work.

So fixing missing dependencies for invalid sourceSets or removing sources from invalid sourceSets fixes the gradle Sync in IntelliJ!

Thanks again for keeping up, much appreciated!