Defining task dependencies for configurations

Help/Discuss
1

I’ve been rewriting on of my gradle plugins that currently defines dependencies in the afterEvaluate block. The main issue is that this creates a lot of spaghetti with static variables being present here and there, and is generally not that configurable anyways. Plus it’s incompatible with the configuration cache.

So, instead of generating the dependencies at configuration time, I now generate them through tasks. This works decently well enough, except with the issue that I don’t know how to get the dependencies attached to the configurations (while I could just add them directly to the JavaCompile tasks, that is a bit involved and wouldn’t fly all too well with IDEs anyways).

In the past (pre-rewrite), I would declare a flatfile maven repository and declare the dependencies using the standard dependency notation. While that approach technically still works, it still requires one to know what the dependencies will be at configuration time (if that’s done in a task, IDEs will be completely oblivious to the dependencies), and also has the issue of IDEs not running the tasks that generate the dependencies by default.

So, my current approach would be to ensure that when a configuration is resolved, the dependent tasks get executed. Unfortunately, something along the way of configurations[“myConfig“].getBuildDependencies().add(myTask)did not seem to have any effect - which I suppose is understandable. Basically, I’m looking for the opposite of myTask.dependsOn(configurations[“myConfig“].getTaskDependencyFromProjectDependency(true, “myTask”)).

2

Can you maybe show an MCVE that shows what you try to achieve?

3

Not exactly sure what you are looking for, so I’ll just reproduce what I have tried so far (in a situation that is extremely detached from reality, but it is enough to highlight what I’m doing):

plugins {
    id 'java'
    id 'java-library'
    id 'maven-publish'
}

task myJar(type: Jar) {
    archiveClassifier = 'myclassifier'
    from "build.gradle"
}

version = "1"

// Method 1: Just include it in the classpath
// Works fine, but Eclipse won't recognize the dependency
/*
compileJava {
    classpath += files(myJar.outputs)
    println(classpath.getFiles())
}
*/

// Method 2: Add myJar to a configuration using PublishArtifacts
// Doesn't work anyways
/*
configurations {
    myConfiguration {
        artifacts.add(objects.newInstance(org.gradle.api.internal.artifacts.publish.ArchivePublishArtifact.class, myJar))
    }
    compileClasspath.extendsFrom(myConfiguration)
}

classes {
    // dependsOn myJar
    doLast {
        println(project.tasks["compileJava"].classpath.getFiles())
        println(configurations["myConfiguration"].artifacts)
        configurations["myConfiguration"].artifacts.forEach {
            println(it.file)
        }
        println(configurations["myConfiguration"].resolve())
    }
}
*/

// Method 3: Flatfile repository approach
// Works great, but fails when importing a project that hasn't been built yet.
/*
repositories {
    flatDir {
        dirs "build/libs"
    }
}

configurations {
    myConfiguration
    compileClasspath.extendsFrom(myConfiguration)
}

// This method blows up if you uncomment the line below, hinting that this method shouldn't be used anyways
myJar.archiveBaseName.set("reproduction-jar")

dependencies {
    myConfiguration ":" + myJar.archiveBaseName.get() + ":1:" + myJar.archiveClassifier.get()
}

compileJava {
    dependsOn myJar
}

classes {
    dependsOn myJar
    doLast {
        println(project.tasks["compileJava"].classpath.getFiles())
        println(configurations["myConfiguration"].artifacts)
        configurations["myConfiguration"].artifacts.forEach {
            println(it.file)
        }
        println(configurations["myConfiguration"].resolve())
    }
}
*/
4

If you can’t split this into two projects, one building the myJar and the other depending on that project, I’d strongly recommend you at least properly model the additional artifact as a feature variant on which you then simply can depend even within the same project.

All methods you showed are quiiite hacky and bad practice. :slight_smile:
You should try a bit less “Ant-y” defining what to do and more “Gradle-y” model the situation and let Gradle do its magic, that will save you a lot of headache on the long run. :slight_smile:

Btw. I also strongly recommend you switch to Kotlin DSL. By now it is the default DSL, you immediately get type-safe build scripts, actually helpful error messages if you mess up the syntax, and amazingly better IDE support when using a good IDE like IntelliJ IDEA or Android Studio (and if it is just for editing the build scripts).

5

About splitting the project, it’s less about not being able to and more about it making no sense whatsoever. I wouldn’t use a subproject just to be able to use implementation gradleApi() after all. In fact, I would go to say that gradleApi() is very close to what I’m seeking to emulate here, just maybe with a bit more configurability? Although I suppose I’d end up where I started if I move away from tasks. Well I guess it’s that or embracing the eclipse-specific hacks. As for how IJ behaves, well - it’ll be how it’ll be.

Not entire sure about the feature variant aspect, but would that even properly get the task executed when the artifact is being resolved?

As for it feeling like Ant - though I’m not sure what exactly you referr to, I’m not old enough to have used Ant after all - and I don’t think you should be selling “magic” to a programmer.

About the kotlin DSL: All that is surprisingly useless when your IDE (eclipse) won’t even recognize a kotlin gradle project as a gradle project in the first place. Besides, 95% of the actual build logic is written as a gradle plugin in Java, the remaining 5% really doesn’t require any of those once one mastered the art of copy & paste.

6

I wouldn’t use a subproject just to be able to use implementation gradleApi() after all.

I don’t really get what you try to say.
That statement of course does not make much sense, but also has not much to do with what I said.
Unless it sounded like that because you left out too much information. :man_shrugging: :slight_smile:

In fact, I would go to say that gradleApi() is very close to what I’m seeking to emulate here, just maybe with a bit more configurability?

Still not really sure what you mean, sorry.
That is a highly internal special-case providing the Gradle API jar with many 3rd-party stuff included, that is provided through massively internal special-casing and should not be a pattern anyone should ever strive to follow.

Not entire sure about the feature variant aspect, but would that even properly get the task executed when the artifact is being resolved?

Should be, unless your description was abstracted too far.
For example something like

val foo by sourceSets.registering
java {
    registerFeature("foo") {
        usingSourceSet(foo.get())
    }
}
dependencies {
    implementation(project(project.path)) {
        capabilities {
            requireFeature("foo")
        }
    }
}

should work just fine.

As for it feeling like Ant - though I’m not sure what exactly you referr to, I’m not old enough to have used Ant after all

Well, in Ant you more program step by step what should happen to build the project.
While Gradle can be used like that, it is by far no near idiomatic, but you should model the project and let Gradle figure out the work it needs to do to build it.

and I don’t think you should be selling “magic” to a programmer.

Ah, sorry, thought you are a developer.
Developers value very much a decent amount of magic that makes their lives easier.
Gradle - or actually almost any build tool - provides many magic so that you write a few lines of build definition and they do the necessary work automatically for you instead of you invoking all the tools and bells and whistles manually.

About the kotlin DSL: All that is surprisingly useless when your IDE (eclipse) won’t even recognize a kotlin gradle project as a gradle project in the first place.

Definitely, but then you might be doing something majorly wrong.
Buildship should support Kotlin DSL Gradle builds just fine.
Eclipse might not provide too much support when editing the build scripts, but neither does it for Groovy DSL.

Besides, 95% of the actual build logic is written as a gradle plugin in Java, the remaining 5% really doesn’t require any of those once one mastered the art of copy & paste.

:ok_hand:

7

I’m afraid I still do not understand the whole variant selection approach.

I currently have

plugins {
    id('sml6') version '0.0.1'
    id 'base'
    id 'maven-publish'
    id 'java-library'
}

task fetchGalim(type: org.stianloader.sml6.FetchGameTask) {
    doLast {
        println "fetchGalim ran"
    }
}

task deobfGalim(type: org.stianloader.sml6.DeobfuscateGameTask, dependsOn: fetchGalim) {
    inputJar = fetchGalim.outputJar
}

// IDE task dependencies
java {
    registerFeature("fakeGameJar") {
        usingSourceSet(sourceSets.create("fakeGameJar") {
            compiledBy(deobfGalim)
        })
        capability("org.stianloader", "fake-game-jar", "1")
    }
}

configurations {
    fakeGameJarApiElements.outgoing {
        artifact(deobfGalim.outputJar) {
            builtBy(deobfGalim)
        }
        capability("org.stianloader:fake-game-jar:1")
    }
}

dependencies {
    compileOnly(project(project.path)) {
        capabilities {
            requireCapability("org.stianloader:fake-game-jar:1")
        }
    }
}

// Verification
afterEvaluate {
    println(configurations["compileClasspath"].resolve())
}

however, the IDEs (Eclipse, IJ Community, and invoking gradle via the wrapper from the CLI) refuse to execute the necessary tasks. Additionally, the IDEs are adamant about not using the declared artifacts, instead preferring to source the dependency from the non-existent classes directory that only exists because features without source sets are not possible.

While the two tasks used here could be migrated to afterEvaluates, later on I’d like to have logic that depends on user-defined artifacts to be resolved - given that resolving configurations at configuration time is slated for removal from what I have read, I’m not for going that way.

Also yes, you are right about buildship supporting kotlin gradle. Seems like even the eclipse ecosystem managed to move forwards in the past two years.

8

Ok, given this more extensive example, feature variants are probably not the right thing to use unless you also decompile the code and put it to a source dir of that source set.

From a cursory look I’d say this feels more like a use-case for an artifact transform, if sml6 also supports that or provides the logic that the tasks use programmatically so that you could use them in an own artifact transform.