Trying to iterate over dependencies down to the "File" level for a cpp-library configuration


(Kent Fletcher) #1

This link -> ArtifactResolutionQuery discusses the ’ ArtifactResolutionQuery’. This link is resolving dependencies in the context of java, but I am trying to do the equivalent dependency resolution for a native builds using cpp-library plugin. I am able to get the name:group:version of a dependency using other means, but I need more information about the dependency such the absolute file path to a dependency where it resides in another project, or in gradle cache dir. So, in the spirit of the link above, and trying to get more details about my dependencies, I have the following DSL.

task resolveCompileSources {
    doLast {
        //configuration.'api'
        def componentIds = configurations.api.incoming.resolutionResult.allDependencies.collect { it.selected.id }

        def result = dependencies.createArtifactResolutionQuery()
                .forComponents(componentIds)
                .withArtifacts(CppComponent)
                .execute()

        for (component in result.resolvedComponents) {
            component.getArtifacts(SourcesArtifact).each { println "Source artifact for ${component.id}: ${it.file}" }
        }
    }
}

Apparently, this may work for java configuration, but apparently not in my native ‘api’ config, because I get the following error message:

* What went wrong:
Execution failed for task ':utilities:resolveCompileSources'.
> Resolving configuration 'api' directly is not allowed

Is there a way I can get file level details for my native dependencies? I’ve seem to have exhausted all ends.

I also investigated -> iterating_over_dependencies_assigned_to_a_configuration to no avail. Because this DSL would suit my needs, but it also doesn’t work…

task iterateResolvedArtifacts {
    dependsOn configurations.api

    doLast {
        configurations.api.each {
            logger.quiet it.absolutePath
        }
    }
}

It yields the exact same error.

* What went wrong:
Execution failed for task ':utilities:resolveCompileSources'.
> Resolving configuration 'api' directly is not allowed

(Shahji) #2

Can we get a comment from the native team on this please. How to get the path in cache for a native api/implementation dependency? How to do it AFTER the dependency has been resolved?

#native


(Kent Fletcher) #3

Yes I’ll make another push for some answers


(Daniel Lacasse) #4

Thanks for this answer, the obvious caveat I can see is the api configuration is both not consumable and not resolvable. It’s like a lifecycle configuration. It’s there as a placeholder to declare dependencies. What you want instead is use one of the more specific dependencies such as cppCompile*, nativeLink*, and nativeRuntime*. Would it be possible to provide a sample of what you are trying to achieve? It seems you are using one of the gradle/native-sample and it would be helpful to have a complete sample to work with.


(Kent Fletcher) #5

Any cpp project in native-samples with dependencies will do either using cpp-application or cpp-library.

For example, I copied and pasted the following task definition (below) to the build.gradle file in -> https://github.com/gradle/native-samples/tree/master/cpp/binary-dependencies

task iterateResolvedArtifacts {
    dependsOn configurations.implementation

    doLast {
        configurations.implementation.each {
            logger.quiet it.absolutePath
        }
    }
}

However, in this build sample, we have an ‘implementation’ configuration where we have applied the ‘cpp-application’, instead of an ‘api’ configuration applying ‘cpp-library’. It to gives a similar error.

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':iterateResolvedArtifacts'.
> Resolving configuration 'implementation' directly is not allowed

Here is the build.gradle in it entirety (with the new task) for native-samples/cpp/binary-dependencies

plugins {
    id 'cpp-application'
    id 'xcode'
    id 'visual-studio'
}

repositories {
    maven {
        url '../repo'
    }
}

application {
    dependencies {
        implementation 'org.gradle.cpp-samples:list:1.5'
    }
}

task iterateResolvedArtifacts {
    dependsOn configurations.implementation

    doLast {
        configurations.api.each {
            logger.quiet it.absolutePath
        }
    }
}

Given your explanation, this tells me that the implementation configuration like the ‘api’ configuration is just a placeholder, and it isn’t consumable and nor resolvable. This is difficult to understand because this leads me to ask you how does gradle consume and resolve it internally? Because somehow, the dependency has to resolve to some file for successful compilation I would imagine. Also, it sound like you are saying we need to declare the dependency differently, i.e., as “cppCompile*”, “nativeLink*”, or “nativeRuntime*”? But if so, how is that done? I don’t see any samples using such a declaration. Ultimately, the motivation behind this post as I expressed in the slack channel is to solve the messy headers issue.


(Shahji) #6

@Daniel_L This was an attempt to work around the “Messy Headers” issue reported here 869. The issue comments has some additional information on what we are trying to do. In short, we want to add subdirectories of the resolved cppHeaders dependency to the compile task’s include directories list.

For e.g. for a resolved headers dependency for say, boost library, gradle adds this path among the list of include directories -

/IC:\Users\shahji\.gradle\caches\5.1-20181114223301+0000\native-dep\5f1c04rxzbuxpk4pudjp1lodi\boost-1.61.0

we would like to include the following to the list -

/IC:\Users\shahji\.gradle\caches\5.1-20181114223301+0000\native-dep\5f1c04rxzbuxpk4pudjp1lodi\boost-1.61.0\algorithm
/IC:\Users\shahji\.gradle\caches\5.1-20181114223301+0000\native-dep\5f1c04rxzbuxpk4pudjp1lodi\boost-1.61.0\align

(Daniel Lacasse) #7

Let’s go through some of the basic of the Gradle’s dependency engine. Each configuration declared can be either consumable, resolvable, or neither. For each project, you can list the dependency configuration with ./gradlew dependencies and you would get something like:

> Task :dependencies

------------------------------------------------------------
Root project
------------------------------------------------------------

api (n)
No dependencies

cppApiElements (n)
No dependencies

cppCompileDebug
\--- project :foo

cppCompileRelease
\--- project :foo

debugLinkElements (n)
No dependencies

debugRuntimeElements (n)
No dependencies

implementation (n)
\--- project foo (n)

mainDebugImplementation (n)
No dependencies

mainReleaseImplementation (n)
No dependencies

nativeLinkDebug
\--- project :foo

nativeLinkRelease
\--- project :foo

nativeRuntimeDebug
\--- project :foo

nativeRuntimeRelease
\--- project :foo

releaseLinkElements (n)
No dependencies

releaseRuntimeElements (n)
No dependencies

The configuration ending with Elements are by convention consumable. This is where you declared the artifact to be exported out of the project. The other ones nativeLink*, nativeRuntime*, and cppCompile* are the configuration that you can resolve for each of the compilation stages. This is what Gradle resolve when it’s building. The *implementation* and api are the configurations that act as lifecycles. All the other configurations extend from those lifecycles so any dependencies added to the lifecycle are being also added the other configuration as they are extending it. The selection of the artifact is done through the attributes.

In this case, what is required is iterating over the right cppCompile* configuration. This will gives only the include paths.

Depending on how this issue is planned to be solved, there may be an alternative as it’s not recommended to reference configuration by name since those changes depending on which dimension participate to the variant building (e.g. build type, operating system, architecture).


(Shahji) #8

@Daniel_L Appreciate the lesson on dependencies. Lot of learning to do to better our self at daily tasks.

This is absolutely not our desired approach. We have been blocked on this for weeks and that called for any solution that would get us moving again.

We will take any better solution that you can propose. What’s on your mind?