Java project with native dependencies with gradle 9

Help/Discuss
1

Hi,

I have a project, which consists of many native .dll/.so files and executable files, which I am moving from ant to gradle.

The project is structured into multiple modules, which are composed differently for different releases. These modules are all gradle subprojects and may produce or require native .dll/.so files to properly work. The native files should be resolved automatically through the dependency graph, so that gradle finds the required binary files accordingly.

My current structure looks similar to the following.

For this I defined a buildlogic plugin with the following definition:

// buildlogic.common
repositories {
    mavenCentral()
}

configurations {
    nativeDependency {
        canBeConsumed = false
        canBeResolved = false
    }
    consumable('nativeElementWindowsX86') {
        attributes {
            attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage, Usage.NATIVE_RUNTIME))
            attribute(Bundling.BUNDLING_ATTRIBUTE, objects.named(Bundling, Bundling.EXTERNAL))
            attribute(OperatingSystemFamily.OPERATING_SYSTEM_ATTRIBUTE, objects.named(OperatingSystemFamily, OperatingSystemFamily.WINDOWS))
            attribute(MachineArchitecture.ARCHITECTURE_ATTRIBUTE, objects.named(MachineArchitecture, MachineArchitecture.X86))
        }
        outgoing {
            artifact project.layout.projectDirectory.dir('windows-x86')
        }
    }
    // And for linux x64 and arm64 in similar fashion
    resolvable('nativeRuntime') {
        extendsFrom nativeDependency
        attributes {
            attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage, Usage.NATIVE_RUNTIME))
        }
    }
}

This plugin is then applied to java projects and a custom c++ projects (with cmake builds). Further I have 3rd party (non maven) dependencies which also require native artifacts and are defined as their own library projects. In the end I have an aggregator plugin, which defines the distribution plugin and puts everything together.

This leads to the following problem: I have issues getting the c++ dependencies to my aggregator project, which generates the final application artifacts.

// buildlogic.cpp
plugins {
    id 'buildlogic.common'
}

tasks.register('clean', Delete) {
    delete 'build'
}


// buildlogic.java
plugins {
    id 'java'
    id 'buildlogic.common'
}

configurations {
    // nativeDependency {
    //     extendsFrom implementation
    // }
    implementation {
        extendsFrom nativeDependency
    }
}


// Project A - c++
plugins {
    id 'buildlogic.common'
    id 'buildlogic.cpp'
}

tasks.register('buildWindowsX86', Sync) {
    from 'lib/windows-x86'
    into 'build/out'
}

artifacts {
    nativeElementWindowsX86 tasks.named('buildWindowsX86')
}


// Project B - java
plugins {
    id 'buildlogic.java'
}

dependencies {
    nativeDependency project(':A')
    implementation 'org.apache.commons:commons-lang3:3.20.0'
}

// Project C - java
plugins {
    id 'buildlogic.java'
}

dependencies {
    implementation project(':B')
}

// Project aggregator
plugins {
    id 'buildlogic.common'
    id 'distribution'
}

configurations {
    module {
        canBeConsumed = false
        canBeResolved = false
    }
    resolvable('moduleRuntime') {
        extendsFrom module
    }
    resolvable('moduleJavaRuntime') {
        extendsFrom module
        attributes {
            attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage, Usage.JAVA_RUNTIME))
        }
    }
    resolvable('moduleNativeRuntime') {
        extendsFrom module
        attributes {
            attribute(OperatingSystemFamily.OPERATING_SYSTEM_ATTRIBUTE, objects.named(OperatingSystemFamily, OperatingSystemFamily.WINDOWS))
            attribute(MachineArchitecture.ARCHITECTURE_ATTRIBUTE, objects.named(MachineArchitecture, MachineArchitecture.X86))
        }
    }
    nativeDependency {
        extendsFrom module
    }
}

dependencies {
    module project(':C')
}

distributions {
    main {
        contents {
            into('lib') {
                // Expected project B, C, commons-lang3

                // This does not work
                // from configurations.moduleRuntime.incoming.artifactView {
                //     withVariantReselection()
                //     attributes {
                //         attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage, Usage.JAVA_RUNTIME))
                //     }
                // }.files
                from configurations.moduleJavaRuntime
            }
            into('bin') {
                // How do I properly resolve this?

                // Expecting some.dll and other.dll in here

                // from configurations.nativeRuntime.incoming.artifactView {
                //     withVariantReselection()
                //     attributes {
                //         attribute(OperatingSystemFamily.OPERATING_SYSTEM_ATTRIBUTE, objects.named(OperatingSystemFamily, OperatingSystemFamily.WINDOWS))
                //         attribute(MachineArchitecture.ARCHITECTURE_ATTRIBUTE, objects.named(MachineArchitecture, MachineArchitecture.X86))
                //     }
                // }.files
                from configurations.moduleNativeRuntime
            }
        }
    }
    // ... other target distributions
}

With the views I get errors, that no matching variant was found.

What am I doing wrong with my setup? Are there better ways to achieve my goal?

Thanks in advance!

2

I got it somewhat do work in the following example: GitHub - RandomName325398/gradle-native-example: Example for gradle 9 native configuration. · GitHub
My issue with the given example was in buildlogic.common, where the following has to be changed:

consumable('nativeElementWindowsX86') {
    extendsFrom nativeDependency
    attributes {
        attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage, Usage.NATIVE_RUNTIME))
        attribute(Bundling.BUNDLING_ATTRIBUTE, objects.named(Bundling, Bundling.EXTERNAL))
        attribute(OperatingSystemFamily.OPERATING_SYSTEM_ATTRIBUTE, objects.named(OperatingSystemFamily, OperatingSystemFamily.WINDOWS))
        attribute(MachineArchitecture.ARCHITECTURE_ATTRIBUTE, objects.named(MachineArchitecture, MachineArchitecture.X86))
    }
    outgoing {
        artifact project.layout.projectDirectory.dir('windows-x86')
    }
}

I’m still not much satisfied, as this requires setting the lenient = true, when adding the artifacts to my distribution. If anyone has something to improve, I would be pleased to hear it!

3

I didn’t check your code so far, but lenient = true is practically never what you want.
It means that all errors that could happen are just swallowed, including things like download errors and so on.
What you probably are after is a componentFilter { it instanceof ProjectComponentIdentifier } to only consider project dependencies and not external ones.

Btw. as you use the new helpers like resolvable and consumable, you should also use dependencyScope instead of setting consumable and resolvable to false manually.