Writing a RuleSource plugin that post-processes .class files

Help/Discuss
1

I am trying to learn how to write a plugin using the new model rules format, but I’m having some trouble mapping what I want to do, to how I must specify it.

In a nutshell, I want to enhance certain classes (using Ebean) before they are added to the classpath for compilation. Therefore, classes in a certain package (e.g. “models”) are compiled before all others, then the Ebean agent is executed on the resulting .class files, resulting in a new set of changed .class files. This final set of .class files should then be added to the compile classpath.

I managed to write a rule to mutate the Java source set in order to exclude the classes that will be compiled separately (based on their path). I managed to create a separate Java source set with only the separate classes. What’s missing is how to run the agent at the right time.

I figure I need to somehow model:

A) That there’s a dependency between the compile configuration and the resulting set of enhanced classes
B) That the resulting set of enhanced classes are generated by first compiling them normally as .java files then running the Ebean agent on them

I’m not sure how I should proceed to do A and B, because I think I’m thinking too much about how to do instead of what should be done. In my mind it should be something similar to chaining two LanguageTransforms. Can such thing be done?

The way it is set-up right now, the separate set of classes to be enhanced is automatically compiled and added to the classpath (i.e., it’s treated as a normal Java source set). I’m thinking maybe I need to isolate the SourceSet (in a separate configuration?). How would I do that?

One way I thought about doing it was to treat the source set as if it were of a separate language (EbeanJavaSourceSet, or something like that), and manually define how it transforms ‘.java’ to ‘.class’ and performs the enhancement. If so, is there a way to reuse the code for compiling ‘.java’ to ‘.class’? (i.e., maybe writing a transform task that somehow uses a Java transform task?)

Am I on the right direction? Is there an easier way to achieve the same result?

2

Hey Janito,

the software model for Java projects is not production ready and will not be developed further. Instead, we will enhance the existing model to provide many of the same benefits.

Now to give you a rough idea how to solve your problem with the current model:

First question: Does Ebean provide an annotation processor? Then you’d just have to change the main java compile task and be done.

If class manipulation is the only way, then you need a new sourceSet that is compiled before the main sourceSet:


sourceSets {
  entities {
    java {
      srcDir 'src/entities/java'
    }
  }
  
  configure([main, test]) {
    compileClasspath += entities.output
    runtimeClasspath += entities.output
  }
}

configurations {
  compile.extendsFrom(entitiesCompile)
  runtime.extendsFrom(entitiesRuntime)
}

Then, you’ll want to call your enhancer as an action right after the compile task for this sourceSet, so that the classes are replaced with the enhanced ones. This will make sure that up-to-date checking will look at your enhanced classes, not the original ones.:

compileEntitiesJava.doLast {
  //call your enhancer here
}

Hope that helps :slight_smile:

Stefan

3

Thanks for the quick answer!

Answering the first question: I don’t believe so, at least AFAIK, however I think it has a runtime enhancer that uses reflection to add the necessary methods when the entity classes are loaded. The problem seems to be that IIRC this could lead to problems if the class loader changes the class load order [1].

I failed to mention I’m attempting to apply the plugin on a Play! framework project, so I was changing the source sets of the PlayApplicationSpec. I erroneously assumed that it would be similar to a generic Java project, sorry :sweat:

Your answer was very enlightening, I still see I have much to learn. If I understood correctly, you added a new source set (normal JavaSourceSet) called “entities”, and by doing so configurations for it were automatically created. When you “configure” the main and test source sets, you’re declaring them to be dependent on the result of the entities source set, correct? The only documentation I found related to the configure call was NamedDomainObjectContainer.configure(), but I’m not sure if (1) it’s the same method and (2) if the parameters specify which elements of the container should have the closure executed on them.

Finally, what I think you did was that you redefined the compile and runtime configurations to extend from the respective configurations of the entities source set. I don’t understand why this is necessary though, don’t the default configurations extend from the main configuration, and therefore automatically include the entities source set as well?

I did understand though that by having the separate source set be a dependency of the main (and test) source sets I can replace the class files after the compileEntitiesJava has executed before they are placed in the classpath to compile the other source sets.

In the Play plugin I believe I can apply a solution very similar to this one. I can create a new source set for the entities, I can hook to the compile task to run the enhancer, and I believe I can update the other source sets (specifically “java” and “scala”) to depend on the output of the entities source set, but I’m not yet certain if I can update the configurations (because if I understood correctly the configuration is a PlayConfiguration that seems to be different than a normal configuration).

If you don’t mind me asking, is there a reason the software model for Java projects won’t be developed further? Is rule-based plugins still the way to go (especially for the long-term)?

Thank you again very much for the reply!

4

Ah, in that case you will indeed have to use rule-based configuration, as the Play plugin is built on that.

Correct.

It’s Project#configure and just a convenience method to apply the same closure to several objects. In this case I wanted to do the same thing to both main and test.

I don’t know what you mean by “default” and “main” in that sentence. We need to make sure that everything your entities need transitively is also visible to the code that uses your entities. That’s why compile needs to extend entitiesCompile.

There will be an in-depth blog post about this decision soon. The short answer is that many of the benefits can be achieved with the “old” model too. And the migration path for people with large builds is just too hard. That’s why we decided to take a step back and instead evolve the old model. In the long run, this might lead to a solution that incorporates many ideas like the rule engine, but in a more incremental way that makes migration easier.

We will of course keep supporting the Play and Native ecosystems that are already built on top of the software model.

You’re welcome :slight_smile:

5

No problem, I’m trying to see if I can map the solution to rules. I haven’t been able to add the outputs from the entities source set to the other source sets though, I tried using entitiesSourceSet.output first, but it complained that there’s no property output for JavaSourceSet. Then I tried:

javaSourceSet.compileClassPath += entitiesSourceSet.buildTask.outputs.files
javaSourceSet.runtimeClassPath += entitiesSourceSet.buildTask.outputs.files

but now it complies that buildTask is null. Do you have any ideas on how to access the output files or how to set the dependency inside a rule?

Ah, now I understand, much simpler than I thought!

Sorry, I think I chose some bad terms. By default I meant the “compile” and “runtime” configurations, and by main I meant “mainCompile” and “mainRuntime”. I think I understand now, it’s not about the classes being in the classpath, it’s about the dependencies of the entities (e.g., the Ebean Jar) being in the runtime, right?

Cool, thanks for the answer. I’ll wait for more info from the blog post :slight_smile:

6

I’m afraid I’m not familiar enough with our Play support to answer that, but I’m sure @sterling or @Gary_Hale can help out.

7

I think I managed to do it, but I had to do it differently. More specifically, I had to create a JvmComponent that generates a Jar file, configure it with the .java files I want from the Play component’s Java source set, then hook an enhancement task after the compilation of the custom component’s source set.

Adding the new Jar as a dependency to the Play tasks was non-trivial. For the Play component’s Java source set, I added it as a library dependency. However, I also needed to do add it as a dependency for the Scala source set, and there I only managed to do that by adding the Jar creation task’s outputs to the classpath of the PlatformScalaCompile task. This seems a bit awkward, though.

I created a plugin to practice my Gradle skills, and although it lacks polish, the rules can be seen by anyone who’s interested: https://github.com/jvff/play-ebean-gradle-plugin/blob/master/src/main/groovy/com/janitovff/play/ebean/gradle/plugins/PlayEbeanGradlePlugin.groovy

There are two things I’m confused about. The first one is if there’s a reason for ScalaLanguageSourceSet to be different from JavaSourceSet, at least regarding dependencies. I can add dependencies to a JavaSourceSet, but to a ScalaLanguageSourceSet I need to manually update the classpath. Does Scala do anything different, or is Scala support not fully model-rule based? With development halted on the software model for Java projects, could the JavaSourceSet dependency handling become deprecated in the future?

The second one is about configurations and the software model domain. Are configurations part of it, or is it something that shouldn’t be used with the model domain? I ask this because I don’t fully understand the play plugin’s configurations. Is it something to help the user to organize dependencies of multiple related source sets, or is it something temporary that will (or should) migrate to the model domain in the future?

Thanks, and sorry about flooding the forum with long posts :confused:

8

Well done @jvff! I was just starting to respond to this thread with an answer similar to what you came up with (create a jvm library and use it as a dependency), but you got there before me.

As far as the scala compile task not getting the appropriate dependencies, adding a library dependency like you have done should work, but I think you’re being thwarted by this heavy-handed piece of configuration: https://github.com/gradle/gradle/blob/04a56fd49290863144d3817548a1bf3e0917f477/subprojects/platform-play/src/main/java/org/gradle/play/plugins/PlayApplicationPlugin.java#L242-L242. That’s essentially setting the classpath to what’s on the play configuration and wiping anything else out. So how to work around that? You should be able to add your library to the play configuration so that it’s part of what gets set:

    @Mutate
    void createTaskToEnhanceClasses(ModelMap<Task> tasks, PlayPluginConfigurations configurations) {
        tasks.create("enhanceEbeanEntities", EnhanceEbeanEntitiesTask) { task ->
            configurations.play.addDependency(task.outputs.files)
        }
    }

That should allow you to keep your dependency on the classpath even with the overwritten classpath.

With respect to your second question, configurations are not implicitly a part of the software model yet, but for the Play support we have bridged some aspects of it over in the form of PlayPluginConfigurations.

9

Hello again, sorry for the late answer. Thanks for the suggestions! I also followed @st_oehme suggestions to make the enhancement task incremental, but to do so I decided to restructure so that the component generates two jars, the ‘default’ Jar (created by the JvmComponent rules) and an ‘enhanced’ jar, with the enhanced version of the classes. The ‘enhanced’ jar is then added as a dependency to the Play configuration.

I created an initial version of the plugin, and recently started using it. So far it seems to work :slight_smile:

The only thing I’m a little in doubt about is that I added the ‘play-java’ module as a dependency to the new component source set, but I hard-coded the version. I’m not sure how I could leverage the PlayPlatformInternal’s getDependencyNotation method to generate the correct module string to add as a dependency. My attempt failed because it said it couldn’t find (map?) the PlayPlatform parameter (missing a Path?). However, even if it did work, would it be unwise to use an internal view in an external plugin?

Again, thank you all very much for all the help!