Gradle doesn't recognize files as dependencies

[crosswind (David Frenzel)]

Hi all,

I feel like I am missing something rather basic. I have two tasks:

1. RenameSourceIncludesTask
2. DynamicBuildTask

1. copies some source files to an upper-case name into a different directory. 2) uses the files from 1) as inputs. The idea is simple: When any of the source files change 1) should re-run. If 2) is executed and 1) is out-of-date, 1) should automatically re-run.

2. and 2) track the file changes independently, i.e., when an input to 1) changes the task is correctly marked out-of-date and when 1) ran, 2) is correctly marked out-of-date because 1) changed something. 1) and 2) are just not dependent on each other unless I specifically tell 2) to dependsOn() on 1).

Both tasks have a ConfigurableFileCollection. The one for 1) is annotated with @OutputFiles and the one for 2) with @InputFiles. During task registration I pass in the same list to both collections when registering them.

How can I get the two tasks to be dependent on each other without specifying it manually?
I have tried using the ConfigurableFileCollection from 1) and passing it to task 2) (instead of passing the same List to both).

public abstract class RenameSourceIncludesTask extends DefaultTask {

    @InputFiles
    public abstract ConfigurableFileCollection getLowerCaseSourceIncludes();

    @OutputFiles
    public abstract ConfigurableFileCollection getUpperCaseTemporarySourceIncludes();

    @TaskAction
    /* Task code */
}

public abstract class DynamicBuildTask extends DefaultTask {

    private final ExecOperations execOperations;
    private Class<?> cobolCompilerExecutableClass;
    private Map<String, String> environmentVariables;
    private ModuleOptions moduleOptions;

    @Inject
    public DynamicBuildTask(ExecOperations execOperations) {
        this.execOperations = execOperations;
    }

    @InputFile
    abstract public RegularFileProperty getBuildFile();

    @InputFiles
    public abstract ConfigurableFileCollection getSourceIncludeDependencies();

    @OutputDirectory
    abstract public RegularFileProperty getOutputDirectory();

    @TaskAction
    /* Task code */
}

List<File> upperCaseSourceDependencyNames = calculateUpperCaseDependencyNames();

TaskProvider<RenameSourceIncludesTask> renameTask = target.getTasks().register(String.format("rename-dependencies-%s", buildFileName), RenameSourceIncludesTask.class, renameSourceIncludesTask -> {
 renameSourceIncludesTask.getLowerCaseSourceIncludes().setFrom(analysisResult.getSourceDependencies());
 renameSourceIncludesTask.getUpperCaseTemporarySourceIncludes().setFrom(upperCaseSourceDependencyNames);
});

target.getTasks().register(String.format("build-%s", buildFileName), DynamicBuildTask.class, task -> {
 task.getBuildFile().set(buildFile);
 task.getOutputDirectory().set(outputDirectory);
 task.getSourceIncludeDependencies().setFrom(upperCaseSourceDependencyNames);
 //task.dependsOn(renameTask); I don't want this
});

I haven’t built a self-contained example yet because I feel like the use case is already rather simple.
Thank you in advance for your help.

Cheers - David

[Vampire (Björn Kautler)]

I don’t want this

That is very good, because practically any explicit dependsOn that does not have a lifecycle task on the left-hand side is a code-smell and bad practice.
Most often it is a sign of not properly wiring task outputs to task inputs but configuring some paths explicitly and thus missing the necessary implicit task dependencies, just like it is in your case.

So yes, in the code you showed this is exactly the problem, as instead of wiring task outputs to task inputs you configure paths explicitly and thus there is no connection.

I have tried using the ConfigurableFileCollection from 1) and passing it to task 2) (instead of passing the same List to both).

How did that look like?
I’d say that would be a proper way to do it.
If that does not bear the necessary task dependency you maybe hit a Gradle bug.

Alternatively you can just set the rename task provider itself as input for the follow-up task, as the files in question are the only outputs of the task and when you configure the task as input, all its outputs are used automatically.

If the rename task would have other outputs, using the output property would be the correct way and should contain the task dependency, but in case there is a bug, you could then also use the rename task provider with .map called to get the intended result.

[Vampire (Björn Kautler)]

Ah, I gave it a quick try, it seems an output property of type ConfigurableFileCollection indeed does not get the task dependency added.

I still think this should be a Gradle bug or at least missing feature.

But as you do (should) not want to break task-configuration avoidance by doing task.getSourceIncludeDependencies.setFrom(renameTask.get().getUpperCaseTemporarySourceIncludes()) anyway, this should actually not be a problem for you.

You anyway here want to use task.getSourceIncludeDependencies.setFrom(renameTask.map(renameTask -> renameTask.getUpperCaseTemporarySourceIncludes())) to not break task-configuration avoidance and thereby preserve the task dependency by using a mapped task provider.

[Vampire (Björn Kautler)]

Or actually, as I said just use the whole task as input as long as that is the only output of the type, so task.getSourceIncludeDependencies.setFrom(renameTask).

I reported a feature request anyway to maybe make ConfigurableFileCollection track the task dependency automatically: `ConfigurableFileCollection`s used as output properties should automatically contain implicit task dependencies · Issue #32311 · gradle/gradle · GitHub

[crosswind (David Frenzel)]

Thank you for the detailed information and creating the issue over in GitHub. I used .map() to create the dependency. That worked flawlessly.

For completeness, I’m posting my working solution here. IntellliJ suggested to directly use the method reference instead of the lambda. When using the lambda, one needs to use a different name because renameTask already exists as a variable but that’s minor.

target.getTasks().register(String.format("build-%s", buildFileName), DynamicBuildTask.class, task -> {
 task.getBuildFile().set(buildFile);
 task.getOutputDirectory().set(outputDirectory);
 task.getSourceIncludeDependencies().setFrom(renameTask.map(RenameSourceIncludesTask::getUpperCaseTemporarySourceIncludes));
});

How did that look like?

I did what would’ve broken task configuration avoidance so I wasn’t using the correct way. Only figured this out after your commend on what I shouldn’t do (don’t use task.get()!).

Ah, I gave it a quick try, it seems an output property of type ConfigurableFileCollection indeed does not get the task dependency added.

The reason I thought this was possible because the docs mention the following here: FileCollection (Gradle API 8.12.1)

A file collection may contain task outputs. The file collection tracks not just a set of files, but also the tasks that produce those files. When a file collection is used as a task input property, Gradle will take care of automatically adding dependencies between the consuming task and the producing tasks.

Since ConfigurableFileCollection implements FileCollection the same should apply over there. Am I reading the docs wrong? Is this valuable information for the issue?

[Vampire (Björn Kautler)]

IntellliJ suggested to directly use the method reference instead of the lambda. When using the lambda, one needs to use a different name because renameTask already exists as a variable but that’s minor.

Yeah, sorry, both correct.
Wrote that from the top of my head.

Since ConfigurableFileCollection implements FileCollection the same should apply over there.

Correct

Am I reading the docs wrong?

Not in what you said, but in what you implied.

Is this valuable information for the issue?

No

If what the docs say would not hold, then task.getSourceIncludeDependencies().setFrom(renameTask.map(RenameSourceIncludesTask::getUpperCaseTemporarySourceIncludes)); and task.getSourceIncludeDependencies().setFrom(renameTask); would not bear a task dependency.

The problem is not the behavior when used as task input.
If a Property type is annotated as task output property, it automatically gets the necessary task dependency for that task.
So if you use the Property itself instead of a mapped task provider, you still get the necessary dependency.
But output properties of type ConfigurableFileCollection to not get such automatic task dependencies and that is the “problem”.