Gradle 9 regression? Cannot inject class file instrumentation task after compileJava

Help/Discuss
1

How should I make my plugin that adds class file instrumentation to the output of classes compiled by compileJava work in Gradle 9?

Before Gradle 9, this was simple and worked:

  • change the compileJava output dir to an intermediate directory
  • use that intermediate directory as input directory for my custom task
  • use the original output directory (main/classes) as the output directory of my custom task
  • finalize compileJava by calling my custom task

This does not work in Gradle 9 anymore because whenever the compileJava output dir is changed, the sourceset main destination dir instantly reflects that change. What happens is that my processed classes are stored in the correct location, but all tasks (for example jar, test) that work on the sourceset main output use the classes from the intermediate dir, which contains the unprocessed classes.

Solutions that do not work:

  • process files in place: breaks task caching because only a single task may produce a specific output when task caching is enabled.
  • have my task copy the generated class files to an intermediate directory before processing: same as above
  • changing the default compile spec: hard coded in a private method
  • changing compiler options: I didn’t try yet, but I think the output dir will be set when the compile task is run
  • just have compileJava output to a new directory, and have my task write the classes write to the standard directory and then reconfigure every Gradle task that consumes sourceset main output to instead use my tasks output (not doable).
  • simply changing the sourceset destination dir after task are created and wired: there’s no method (like setClassesDir()) to do that.

What is the recommended way to do this in Gradle 9, or does this simply not work?

2

Before Gradle 9, this was simple and worked:

Not really.
It maybe seemed to work, or worked in a very narrow case in your build.
But if you for example also had other tasks that wired the compileJava outputs to their inputs, those would get the original compilation result, not your instrumented one and so on.

What happens is that my processed classes are stored in the correct location, but all tasks (for example jar, test) that work on the sourceset main output use the classes from the intermediate dir, which contains the unprocessed classes.

Yeah, well, seems the wiring was improved to work more correctly and fix the loophole you used in your build. :smiley:

What is the recommended way to do this in Gradle 9, or does this simply not work?

What you probably always should have done is to not have a separate task, but add a doLast action to the compileJava task, so that your instrumentation is part of the task execution, which means you can modify the class files in-place as you are still operating at the execution of the task.

Maybe another alternative - but I don’t know whether it will work or whether it has other quirks - could be to have an additional compile task that compiles the files to a custom location, configure the “normal” compile task to not have any inputs anymore, have your instrumentation task process the output of the additional compile task, register your compile task output as output of the source set.

3

Is it common to have tasks work on the output of the compile task? All other Plugins I use that consume the class files use the source set output to find those.

I will try adding the doLast thing instead and report back, thank you.

4

Hi @Vampire,

this actually seems to work. I was a bit skeptical that when using doLast() like this, my instrumentation class would not be cacheable anymore, because it’s still a task and I expected it to run into the problem mentioned in the first bullet point above. Instead the build cache now is filled with the result of compilation + instrumentation. At least, that’s how I understand it.

What I ended up doing is instead of registering and wiring an explicit class is just doing instrumentation like this:

        compileJavaTask.doLast(task -> instrumentClasses(extension, compileJavaTask));

So far everything looks good, I tested with 8.14.3 and 9.0.0-rc-3:

  • after running the compileJava task, the classes directory contains the instrumented classes
  • compileJava results are loaded from the build cache, the instrumented classes are uses

I still need to make sure that changing the instrumentation configuration has to invalidate the compileJava results.

And there’s one thing that won’t work anymore: when the instrumentation configuration changes, both the compilation and instrumentation have to be run whereas before, the compilation result was loaded from the build cache and only the instrumentation was re-run. But that’s a minor thing I can live with.

So I’ll clean up the remaining bits and think this is solved.

Thanks once again,
Axel

1 Like
5

Is it common to have tasks work on the output of the compile task? All other Plugins I use that consume the class files use the source set output to find those.

Well, heavily depends.
Most use-cases should probably use the source set output, yeah, as it also contains the class files for other languages like Kotlin, Groovy, Scala, …
But maybe there are also use-cases that need to operate on only the class files originating from Java code. :man_shrugging:

But anyway the source set output should be coupled to the compile task output, which it seems was not the case but is now from what you said.

I was a bit skeptical that when using doLast() like this, my instrumentation class would not be cacheable anymore, because it’s still a task and I expected it to run into the problem mentioned in the first bullet point above.

Your first bulletpoint correctly says that two tasks must not have overlapping outputs, let alone modify the outputs of another task in-place. But by adding a doLast action, this is exactly not the case, as the action is part of the task action, so is done before the task finishes and thus before the result is considered for caching and at the same time your actions source becomes part of the cache key by being part of the runtime classpath for the task.

One thing to consider is, that your instrumentation action has to cope with incremental compilation.
If you only change one source file, only that (and classes depending on its API) are recompiled so your instrumentation needs to handle this, so that for example only changed outputs are instrumented, or you need to prevent incremental compilation, or you might recognize that a class file already has your instrumentation and doesn’t apply it again, or something like that.

I still need to make sure that changing the instrumentation configuration has to invalidate the compileJava results.

Yes, please test, but as I said, by adding that action the action implementation should become part of the task classpath and thus part of the inputs of the task and thus make the task out-of-date and the cache key different when the implementation of the action changes.

And there’s one thing that won’t work anymore: when the instrumentation configuration changes, both the compilation and instrumentation have to be run whereas before, the compilation result was loaded from the build cache and only the instrumentation was re-run. But that’s a minor thing I can live with.

Yes, that’s correct.
If that is really an issue, you can probably do dirtier tricks like doing actions.clear() to remove all existing actions, register an additional task that does the actual compilation and thus could be cached separately and then use that as input to the manipulated task, copying the other tasks output to the destination and doing the instrumentation. But it is highly questionable whether there is enough performance gain to warrant that added complexity.

6

Hi,

thanks for the detailed explanation. It works flawlessly now, although the change to automatically change the sourceset output dir has already been reverted by the Gradle devs as they said this change should go through the usual deprecation process first it is feared I am not the only one running into this.

And I can live with the last point, it is not really an issue.

1 Like