Dependency substitutions and transitivity

Help/Discuss
1

When using dependency substitutions, what is the expectation regarding transitive dependencies of the swapped dependency?

configurations.myConfig.resolutionStrategy {
    dependencySubstitution {
        substitute project( 'hibernate-testing') with project( 'hibernate-testing-jakarta' )
    }
}

dependencies {
    myConfig project( 'hibernate-testing') 
}

Given this build, the reference to hibernate-testing as a dependency in the myConfig Configuration will be substituted with hibernate-testing-jakarta. Will the transitive dependencies of hibernate-testing-jakarta be included in myConfig when it gets resolved?

It seems like this does not happen, but my actual example is much more complicated and just wanted to see if I have the correct expectation here

2

Yes, they should. The hibernate-testing subtree should be replaced by the hibernate-testing-jakarta tree.
If they aren’t, can you create a small, self contained example that demonstrates what you’re seeing?

3

Thanks for the reply @Chris_Dore!

A simple example is not really possible. This is a plugin I am testing and I see this problem in its TestKit tests. The hibernate-testing-jakarta project is “virtual” - the plugin takes hibernate-testing, runs Jakarta’s JakartaTransformer against it and exposes the transformation as hibernate-testing-jakarta.

hibernate-testing is a Java project. hibernate-testing-jakarta is effectively a Java project though the plugin simply tries to set up everything like the java or java-library would have (creates Configurations, POM, etc). It is quite likely that I am just not setting up the metadata (POM) for the hibernate-testing-jakarta project properly from the plugin

If you can spare some cycles, the plugin is here: GitHub - hibernate/jakarta-transformer-plugin. The ShadowSharedTesting test deals with this situation. The test project set-up is:

  1. real = the “real” main project
  2. real-testing = test fixtures
  3. real-jakarta = transformed version of real
  4. real-testing-jakarta = transformed version of real-testing

real depends on real-testing for test compilation. real-testing depends on real for compilation

Both (3) and (4) are virtual projects which are meant to “shadow” (1) and (2) respectively. So the plugin gets applied to (3) and (4) and tries to mimic the (1) and (2) relationship.

  1. The test ShadowSharedTesting#jakartaTestingTest illustrates the problem.
  2. If you run ShadowSharedTesting#pomGenerationTest, you can see the POM generated into the build dir. Again, I am not sure that the POM for the 2 shadow projects are right. But I am also not sure that the POM is what actually gets used when defining a project-dependency.
  3. You can also run ShadowSharedTesting#showDependenciesTest to see the dependency tree that should be applied to test-runtime.

The output from ShadowSharedTesting#showDependenciesTest is

testRuntimeScope
+--- javax.persistence:javax.persistence-api:2.2 -> jakarta.persistence:jakarta.persistence-api:3.0.0
+--- org.junit.jupiter:junit-jupiter-engine:5.7.0
|    +--- org.junit:junit-bom:5.7.0
|    |    +--- org.junit.jupiter:junit-jupiter-api:5.7.0 (c)
|    |    +--- org.junit.jupiter:junit-jupiter-engine:5.7.0 (c)
|    |    +--- org.junit.platform:junit-platform-engine:1.7.0 (c)
|    |    \--- org.junit.platform:junit-platform-commons:1.7.0 (c)
|    +--- org.apiguardian:apiguardian-api:1.1.0
|    +--- org.junit.platform:junit-platform-engine:1.7.0
|    |    +--- org.junit:junit-bom:5.7.0 (*)
|    |    +--- org.apiguardian:apiguardian-api:1.1.0
|    |    +--- org.opentest4j:opentest4j:1.2.0
|    |    \--- org.junit.platform:junit-platform-commons:1.7.0
|    |         +--- org.junit:junit-bom:5.7.0 (*)
|    |         \--- org.apiguardian:apiguardian-api:1.1.0
|    \--- org.junit.jupiter:junit-jupiter-api:5.7.0
|         +--- org.junit:junit-bom:5.7.0 (*)
|         +--- org.apiguardian:apiguardian-api:1.1.0
|         +--- org.opentest4j:opentest4j:1.2.0
|         \--- org.junit.platform:junit-platform-commons:1.7.0 (*)
\--- project :real-testing -> project :real-testing-jakarta

So it would seem that the transitive deps from real-testing-jakarta are not being used. The classpath applied to Test does not even include real-testing-jakarta for some reason, even though it is based on this testRuntimeScope Configuration:

final FileTransformationTask shadowJarTask = ...
testTask.setClasspath(
        testRuntimeScope
                .plus( targetProject.files( shadowJarTask.getOutput() ) )
                .plus( targetProject.files( javaTransformationTask.getOutput() ) )
                .plus( targetProject.files( resourcesTransformationTask.getOutput() ) )
);

As shown above, testRuntimeScope includes real-testing-jakarta (via substitution). But the testTask’s classpath at runtime is:

############################################################
 `:real-jakarta:test` task classpath...
############################################################
   -> /tmp/.gradle-test-kit-sebersole/caches/modules-2/files-2.1/jakarta.persistence/jakarta.persistence-api/3.0.0/affc7884a85b6876d438a88b5d21ea29b1cc2dd8/jakarta.persistence-api-3.0.0.jar
   -> /tmp/.gradle-test-kit-sebersole/caches/modules-2/files-2.1/org.junit.platform/junit-platform-engine/1.7.0/eadb73c5074a4ac71061defd00fc176152a4d12c/junit-platform-engine-1.7.0.jar
   -> /tmp/.gradle-test-kit-sebersole/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter-api/5.7.0/b25f3815c4c1860a73041e733a14a0379d00c4d5/junit-jupiter-api-5.7.0.jar
   -> /tmp/.gradle-test-kit-sebersole/caches/modules-2/files-2.1/org.junit.platform/junit-platform-commons/1.7.0/84e309fbf21d857aac079a3c1fffd84284e1114d/junit-platform-commons-1.7.0.jar
   -> /tmp/.gradle-test-kit-sebersole/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter-engine/5.7.0/d9044d6b45e2232ddd53fa56c15333e43d1749fd/junit-jupiter-engine-5.7.0.jar
   -> /tmp/.gradle-test-kit-sebersole/caches/modules-2/files-2.1/org.apiguardian/apiguardian-api/1.1.0/fc9dff4bb36d627bdc553de77e1f17efd790876c/apiguardian-api-1.1.0.jar
   -> /tmp/.gradle-test-kit-sebersole/caches/modules-2/files-2.1/org.opentest4j/opentest4j/1.2.0/28c11eb91f9b6d8e200631d46e20a7f407f2a046/opentest4j-1.2.0.jar
   -> /home/sebersole/projects/playground/jakarta-transformer-plugin/build/tmp/testKit/testKit803615492/shadowShared/real-jakarta/build/libs/real-jakarta-1.0.0.jar
   -> /home/sebersole/projects/playground/jakarta-transformer-plugin/build/tmp/testKit/testKit803615492/shadowShared/real-jakarta/build/classes/java/test
   -> /home/sebersole/projects/playground/jakarta-transformer-plugin/build/tmp/testKit/testKit803615492/shadowShared/real-jakarta/build/resources/test
############################################################

Which is verbose, but just shows that the real-testing-jakarta project is not included. I presume because it is not set up as a project the way dependency resolution needs it to be.

Not sure if it is important, but wanted to point out that in order to handle both java and java-library I use compileClasspath, runtimeClasspath, testCompileClasspath and testRuntimeClasspath as the basis for the deps of the virtual projects (as opposed to directly referring to implementation, …).

4

I suspect the issue is around the Helper.shadowConfiguration area. When the configurations are cloned the variant/attribute metadata is not carried over. Due to the lack of matching variants I think Gradle is picking the default configuration of the virtual projects, which has no dependencies. If the cloned configs had the correct variant attributes then Gradle should pick the correct configuration with the cloned dependencies.

Some info about how Gradle picks configurations/variants; Understanding variant selection

5

I’m going to have to dig around in variants. I’ve been meaning to research them a bit more anyway for Hibernate’s published artifacts.

When you say that “the variant/attribute metadata is not carried over” when the Configuration is cloned, how would I do that?

6

A Configuration has an AttributeContainer accessible via Configuration.getAttributes() and Configuration.getOutgoing().getAttributes(). These may return a reference to the same container, I cannot recall.

Configurations also have a collection of variants accessible via Configuration.getOutgoing().getVariants(). Each variant has an AttributeContainer via getAttributes().

I’m not sure how the configuration’s attribute container(s) relate to those of the variants. I tried to figure that out once before but it was a long time ago and I recall getting confused by my tests. I know, that doesn’t sound encouraging :slight_smile: but I think if you copy the right attributes and/or variants with their attributes then things will works out. I’m certainly not an expert on this so I’m being a bit vague.

7

I’ll try copying over the attributes and see where that gets me.

Maybe it would be easier to just have my plugin apply the java / java-library plugin on the virtual project and just use the Configurations built by the Java(Library)Plugin. Its an odd mapping though since the virtual project has no sources.

Thanks a ton @Chris_Dore

8

Little different error after copying over attributes. Did not seem to be any variants in :real:compileClasspath, :real:runtimeClasspath, :real:testCompileClasspath or :real:testRuntimeClasspath

Execution failed for task ':real-jakarta:test'.
> Could not resolve all dependencies for configuration ':real-jakarta:testCompileScope'.
   > Could not resolve project :real.
     Required by:
         project :real-jakarta > project :real-testing-jakarta
      > The consumer was configured to find attribute 'org.gradle.category' with value 'library', attribute 'org.gradle.usage' with value 'java-api', attribute 'org.gradle.dependency.bundling' with value 'external'. However we cannot choose between the following variants of project :real-jakarta:
          - compileScope
          - testCompileScope
        All of them match the consumer attributes:
          - Variant 'compileScope' capability org.hibernate.build.gradle.jakarta:real-jakarta:1.0.0 declares attribute 'org.gradle.category' with value 'library', attribute 'org.gradle.dependency.bundling' with value 'external', attribute 'org.gradle.usage' with value 'java-api'
          - Variant 'testCompileScope' capability org.hibernate.build.gradle.jakarta:real-jakarta:1.0.0 declares attribute 'org.gradle.category' with value 'library', attribute 'org.gradle.dependency.bundling' with value 'external', attribute 'org.gradle.usage' with value 'java-api'
        The following variants were also considered but didn't match the requested attributes:
          - Variant 'runtimeScope' capability org.hibernate.build.gradle.jakarta:real-jakarta:1.0.0 declares attribute 'org.gradle.category' with value 'library', attribute 'org.gradle.dependency.bundling' with value 'external':
              - Incompatible because this component declares attribute 'org.gradle.usage' with value 'java-runtime' and the consumer needed attribute 'org.gradle.usage' with value 'java-api'

So if I understand correctly, resolution was unable to select compileScope and testCompileScope because they both matched variant selection? No idea what this means really :wink: