compileClasspath and runtimeClasspath under sourceSets are not picking up dependencies(*.java/*.class/*.jar) similar to (javac) java compiler behavior

I am trying to place java sources and/or classes under compileClasspath of the custom sourceSet mds that I have created, the java sources and/or classpath are supposed to be dependencies that are provided/made available.

Reason for this config: They can be used(in case of classes) or compiled into classes(in case of sources) only when they are being depended on, otherwise the actual java sources in srcDirs only needs to be compiled into classes for the application to run.

I am aware that the custom sourceSet mds in the gradle configuration phase creates new implicit configurations namely mdsCompileOnly, mdsRuntimeOnly, mdsCompileClasspath, mdsRuntimeClasspath, mdsImplementation, mdsTestCompileOnly, mdsTestRuntimeOnly, mdsTestCompileClasspath, mdsTestRuntimeClasspath, mdsTestImplementation

Adding the dependencies directly to the mdsImplementation/mdsCompileClasspath solves the compilation issue, but I want to understand why it can’t be done otherwise. I am not pursuing this way, because this causes duplicate declaration of dependencies differing only by their configuration.

I am aware that a new configuration mds can be created and let that configuration mdsImplementation extendsFrom(implementation), but not pursuing this path either.

`The use-case that we are particularly interested in is use-case-4
(including *.java sources into classpath so as to be made available for compilation (provided)).
Reference: oracle javac

use case 1:

  • libraries (*.jar) sitting on dependency (implementation) of main sourceSet can be shared by this custom sourceSet mds by including the main’s compileClasspath into the compileClasspath for java sources in the custom sourceSet mds.

expectation: any/all library (*.jar) dependencies that are resolved as dependencies of sourceSet main should now be available for the compilation of custom sourceSet mds.
reality: works as expected except when using using few notations (are these invalid?).

plugins {
    id 'java'
    id 'application'
}

sourceSets{
    mds{
        java{
            srcDirs = ['src/mds']
            compileClasspath += sourceSets.main.compileClasspath
            runtimeClasspath += sourceSets.main.runtimeClasspath
        }
    }
}


dependencies {
      // single jar files included as is
    implementation files('lib/junit-4.13.1.jar')                  // <-- WORKS
    implementation files('lib/hamcrest-core-1.3.jar')             // <-- WORKS

      // Root directory which contains the jars junit-4.13.1.jar, hamcrest-core-1.3.jar
//    implementation files('lib')                                   // <-- DOESN'T WORK
//    implementation files(['lib'])                                 // <-- DOESN'T WORK

      // using fileTree expansion by specifying flat root dir (lib) containing all dependent jars
    implementation fileTree(include: ['*.jar'], dir: 'lib')  // <-- WORKS
}

use case 2:

  • *.class files sitting on dependency (implementation) of main sourceSet can be shared by this custom sourceSet mds by including the main’s compileClasspath into the compileClasspath for java sources in the custom sourceSet mds.

expectation: any/all *.class files that are resolved as dependencies of sourceSet main should now be available for the compilation of custom sourceSet mds.
reality: doesn’t work at all using any of the following notation.

plugins {
    id 'java'
    id 'application'
}

sourceSets{
    mds{
        java{
            srcDirs = ['src/mds']
            compileClasspath += sourceSets.main.compileClasspath
            runtimeClasspath += sourceSets.main.runtimeClasspath
        }
    }
}


dependencies {
      // Root directory which contains the extracted files(*.class) from junit-4.13.1.jar, hamcrest-core-1.3.jar
      // Note: File/Dir structure is preserved
//    implementation files('lib_dir')                               // <-- DOESN'T WORK
//    implementation files(['lib_dir'])                             // <-- DOESN'T WORK

      // using fileTree expansion by specifying structured root dir (lib_dir) containing all extracted contents (*.class) from the dependent jars 
//    implementation fileTree(include: ['**/*.class'], dir: 'lib_dir')  // <-- DOESN'T WORK
}

use case 3:

  • libraries(.jar) and/or classes (.class) added via compileClasspath of the mds sourceSet’s are not being detected by the compilation unit.

expectation: any/all library libraries(.jar) and/or classes (.class) that are added into the compileClasspath of the respective sourceSet should be available for compilation but not necessarily be compiled unless required by actual sources set for compilation using srcDirs.
reality: doesn’t work with most of the following notations.

plugins {
    id 'java'
    id 'application'
}

sourceSets{
    mds{
        java{
            srcDirs = ['src/mds']

      // single jar files included as is
//            compileClasspath += files('lib/junit-4.13.1.jar')                 // <-- DOESN'T WORK          
//            runtimeClasspath += files('lib/hamcrest-core-1.3.jar')            // <-- DOESN'T WORK
//            compileClasspath += files(['lib/junit-4.13.1.jar'])               // <-- DOESN'T WORK
//            runtimeClasspath += files(['lib/hamcrest-core-1.3.jar'])          // <-- DOESN'T WORK

      // Root directory which contains the jars junit-4.13.1.jar, hamcrest-core-1.3.jar
//            compileClasspath += files('lib')                                  // <-- DOESN'T WORK
//            runtimeClasspath += files('lib')                                  // <-- DOESN'T WORK
//            compileClasspath += files(['lib'])                                // <-- DOESN'T WORK
//            runtimeClasspath += files(['lib'])                                // <-- DOESN'T WORK

      // using fileTree expansion by specifying flat root dir (lib) containing all dependent jars
            compileClasspath += fileTree(includes: ['*.jar'], dir: 'lib')       // <-- WORKS
            runtimeClasspath += fileTree(includes: ['*.jar'], dir: 'lib')       // <-- WORKS
            
      // Root directory which contains the extracted files(*.class) from junit-4.13.1.jar, hamcrest-core-1.3.jar
      // Note: File/Dir structure is preserved
//            compileClasspath += files('lib_dir')                              // <-- DOESN'T WORK
//            runtimeClasspath += files('lib_dir')                              // <-- DOESN'T WORK
//            compileClasspath += files(['lib_dir'])                            // <-- DOESN'T WORK
//            runtimeClasspath += files(['lib_dir'])                            // <-- DOESN'T WORK


      // using fileTree expansion by specifying structured root dir (lib_dir) containing all extracted contents (*.class) from the dependent jars 
//            compileClasspath += fileTree(includes: ['**/*.class'], dir: 'lib_dir')    // <-- DOESN'T WORK
//            runtimeClasspath += fileTree(includes: ['**/*.class'], dir: 'lib_dir')    // <-- DOESN'T WORK
        }
    }
}

use case 4:

  • java sources (.java) and/or libraries(.jar) and/or classes (*.class) added via compileClasspath of the mds sourceSet’s are not being detected by the compilation unit.

expectation: any/all library java sources (.java) and/or libraries(.jar) and/or classes (*.class) that are added into the compileClasspath of the respective sourceSet should be available for compilation but not necessarily be compiled unless required by actual sources set for compilation using srcDirs.
reality: doesn’t work with any notation shown in above use-cases.

In addition, if someone could provide reason to why some of the other use-cases aren’t working as expected would be useful for my understanding.

Issue has been reported to gradle developers but the response was “Will not be fixed/Working as expected”, please follow up on compileClasspath in sourceSets doesn't work same as classpath option in javac · Issue #25171 · gradle/gradle · GitHub