The mrJar Plugin v0.0.11 has been released — Modular MRJAR Files Made Easy

Gradle issue #10046reported by @chrisdennis a few weeks ago — sparked an idea for a plugin. Earlier today I published the mrJar plugin that fulfills the main requirement requested in that issue. Plus it partially supports parts of the others:


support for:

  • Compiling the needed class-files.
  • Executing unit tests across [some of the] JDK versions.

The plugin’s implementation of @chrisdennis’ list of requirements is still not 100% feature complete. Yet. I’d say it’s maybe %70-80% right now. The same thing goes for the level of my Gradle knowledge at this point (with my Gradle knowledge numbers being a lot lower :disappointed: ). But I intend to fill the gaps in both eventually.

I have established, nevertheless, that the mrJar plugin is satisfactorily usable even at this early v0.0.1 stage. During development of the plugin, I applied the plugin to its own project. It had no problem turning itself into a modular MRJAR-assembled plugin. It was as easy as:

plugins{ 
    ...
    id 'com.lingocoder.mrjar' version '0.0.1'
    id 'java'
    id 'groovy' /* I used Groovy in my project; But you don't have to in yours */
    ...
}
...
dependencies { 
    /* <mrjar plugin project's dependencies> */
}
...
mrjar {
    /* The releases property is mandatory. The values must be Gradle's version enum types   */ 
    releases [JavaVersion.VERSION_1_9, JavaVersion.VERSION_12, ]
}
...

What the mrjar extension does is tell the plugin which Java releases your artifact will support. With that in your build script, you just simply need to run the one single task that the plugin exports:

$ ./gradlew :mrinit

In my example the plugin’s lone :mrinit task created source directories src/main/java9 and src/main/java12 in my project directory.

I then simply added my release-specific source code to the respective release’s source folder. That source code could, optionally, include a module-info.java source file.

Just an aside: My particular use case called for modularity. Your own use case might not necessarily need to support Java 9+ modules. It is a valid use case to support just a Multi-Release Jar that has no module-info.java descriptors. The mrJar plugin can do that for you.

TL;DR: As an end user of the mrJar plugin, there were only two extra steps I needed to do to have Gradle assemble my project as a modular MRJAR artifact:

  1. fill in the properties of mrjar extension
  2. execute ./gradlew :mrinit

After that, I simply carried on the normal steps of the typical test/develop/build cycle. Then when I was eventually ready to publish the plugin to Gradles Plugins Portal, I just did it the same way I normally do it:

./gradlew publishPlugins

Et viola! Who knew dog food was so tastey? :wink: I share more detailed usage examples on the project’s site.

Naturally, there will be things that during development of the mrJar plugin I might have either overlooked, incorrectly assumed or just plain had no clue about. To fill those gaps sooner, I’m reaching out to the community to get feedback on usage of the plugin.

Before I started development of the plugin, I did some analysis/research to establish what was feasible. From that, I learned that these guys know more than a thing or two about MRJARS and/or Java 9 modules:

So I’m requesting a favor from you all (and the Gradle community in general): Please give the usage of the mrJar plugin the once over? Apply the mrJar plugin in your project, see if it works for you, and share your feedback and suggestions? Please? TIA.

1 Like

Two administrative suggestions:

  1. can you share the source on github, or somewhere else?
  2. can you use normal links instead of bit.ly links on your site? It’s nice to see the actual target of a link before one clicks on it, and I can’t imagine that a normal website has a character limit. Maybe you use bit.ly for link usage tracking, but a) is it necessary? & b) if so, is there some other way to achieve link usage tracking while still having the actual URL visible from your site?

I’ll try to look at the plugin later.

I second @rgoldberg’s comments.

Like I said on the Slack channel, I doubt this will solve the problem for this use-case, but the contribution is much appreciated nonetheless.

I will take this plugin for a spin once you publish the source on github or something.

@rgoldberg :+1: Great suggestions!

@lingocoder I get this error when applying your plugin:

A problem occurred configuring root project 'mrjartest'.
> d != java.lang.String
...
...
Caused by: java.util.IllegalFormatConversionException: d != java.lang.String
        at com.lingocoder.plugin.mrjar.TestTaskConfigurator.configureTests(TestTaskConfigurator.java:44)
        at com.lingocoder.plugin.mrjar.MrJarPlugin.lambda$apply$0(MrJarPlugin.java:91)
        at org.gradle.configuration.internal.DefaultListenerBuildOperationDecorator$BuildOperationEmittingAction$1$1.run(DefaultListenerBuildOperationDecorator.java:154)
...

Awesome @siordache. Thanks. Can you share the precise steps for me to reproduce that? Please?

Many thanks :slight_smile:

Ah! Spotted it. Thanks.

'Twas an orphaned embedded String format specifier (%d). Leftover from when I — at the very last minute — replaced all the System.out.printf("...%s...%d ....", "foo", 20) calls with their SLF4j Logger.debug("...{}..{}...". "foo", 20) equivalents.

I was making those changes hastily before I pushed the first release. And I wasn’t being very eagle-eyed in my haste. I’ll fix that real quick and have v0.0.2 published shortly. Sincere thanks @siordache for catching that :slight_smile:

As a bug bounty, I will share with you, @siordache, this work around that will bypass that particular line in the code: Make sure before you apply mrJar that you have JAVA_8 and JAVA_9 environment variables set to the home directories of those two required JDK installations.

Setting those environment variables should be enough for now to avoid that exception; until I can get the fix up to the portal.

I haven’t had time to document that requirement anywhere yet. So thanks again for reminding me to put it on my TODO list :+1:

I haven’t yet used the plugin, but, from looking at the usage instructions, I think that mrjar {} should be optional.

For the setup, since the the mrinit task only needs to run once, you should be able to specify the values for it in the command line. If any values exist in the build script, the command line should override them, and an override warning should be displayed.

For modifying the behavior of the jar task, if mrjar {} doesn’t exist, then your plugin should just look for directory structures that match its conventions, and automatically create a multi-release jar from them. e.g., if src/main/java9 & src/main/java12 exist, it will create a multi-release jar with code for those versions, even if mrjar {} doesn’t exist. Convention over configuration can prevent sync errors.

Also, if anything matches the conventions in the directory structure, but is left out of a mrjar {} setup, then you might want to output a warning (and maybe provide a way to silence such warnings, too). Obviously, your plugin should output an error if something is in mrjar {} but missing from the directory structure, which I assume it already does.

One note: once you get everything working, you might want to create a PR that just modifies the actual Jar task class to support this directly in Gradle core. You’d probably want to have the Gradle team look over your plugin code first, before you spend time integrating it with the core Jar task, for which hosting your code on github would be useful.

Make sure before you apply mrJar that you have JAVA_8 and JAVA_9 environment variables set to the home directories of those two required JDK installations .

Why do you need multiple JDK installations?

Hey @siordache, v0.0.2 with the fix is available now. I think I got all those little suckers this time :slight_smile:

Sincere thanks again for bringing them to my attention.

Hey thanks @rgoldberg for the feedback. You’ve given me a lot of food for thought. And very valuable morsels they are. Sincerely appreciated :slight_smile:

Would you accept: They’re there for sentimental reasons HaHa!

No. But seriously. Very often, I’ve seen answers given here in the forums and elsewhere, from Gradle core devs addressing similar questions. I hope my answer is similarly sufficient: It’s an implementation detail :slight_smile:

I don’t know why you’d need more than one JDK to build a multi-release jar. Shouldn’t you only need a single JDK 9+ that supports at least the newest release of java that will be included. Then you could just vary the target jvm via the --release <version> option.

Does your plugin set the --release <version> option for JavaCompile? As long as each source directory for a different java version is in its own source set, then you could have a JavaCompile for each, and set its --release to the appropriate version.

The standard Gradle location for a separate java9 source set would normally be src/java9, which should be better than src/main/java9, because following the convention will easily allow you to compile on a per-java-target-version basis source from other languages, e.g. Kotlin. This would allow you to have src/java9/java, src/java9/kotlin, etc. as per usual.

You should also automatically set jvm versions for other compilers, like setting KotlinCompile#kotlinOptions.jvmTarget. Obviously, supporting additional languages would have to avoid always applying every additional language’s plugin. Supporting mrjars directly from Gradle’s core jar task would more easily allow every language plugin to register support for setting up the correct jvm target for its compile task.

Hi @lingocoder, thanks for mentioning me, but I won’t be of help here.

Having read Cedric’s post on MRJARs, I’m not a fan of them either. When I contributed separate module-info.java compilation to Gradle Modules Plugin, I purposefully decided not to use MRJARs.

Still, I appreciate your contribution to the JPMS & Gradle ecosystem, and wish your plugin all the best!

@tlinkowski Does Gradle currently have any support for easily building & depending on JVM-version variants of an artifact / module? (I don’t know the difference between artifact & module).

Would these variants have the same coordinates and file name, or would one or both of them have the JVM version directly included?

Would these variants require a Gradle module descriptor?

Would a repository have to explicitly add support for variant resolution, or is that something that Gradle can handle without requiring changes to repository software?

How would other build tools access the variants? Does Gradle provide a standalone library to help add variant dependency support to other tools?

Thanks.

Hi Ross,

I’m afraid I don’t really know how to answer most of your questions.

To the best of my knowledge, Gradle has no native support for depending on JVM-version variants of a module (as I understand it, a module has many artifacts, typical being “jar” artifact, “sourcesJar” artifact, etc. - with N variants, there would probably be N times as many artifacts)

The closest I found to this when I was exploring possible architectures for my UniJ project was https://github.com/uklance/gradle-java-flavours by @Lance_Java. Perhaps Lance can answer some of your questions.

I, after all, decided upon a different (Java-service-based) architecture for UniJ, because I believed it would be easier to maintain in a library, and I was afraid about IDE (in my case, IntelliJ) support for such flavors.

Thanks for your kind words of encouragement, @tlinkowski. I sincerely appreciate it :slight_smile:

It’s interesting, isn’t it @tlinkowski, how one person’s (or one organization’s) opinion can lead one person’s view of something toward one direction? And that same persorganization’s opinion can conversely lead the next person’s opinion toward a completely different view of the exact same issue?

That is: The same blog post that turned you off MRJAR Files, made me more intrigued to want to learn as much about them as I possibly could.

There always was something about that blog post you linked to, that made me wonder if the curiously intense anti-MRJAR opinion it gives, might possibly have something to do with unspoken, non-technical reasons.

I mean, if I — somebody who is relatively green as far as Gradle knowledge goes — can implement a satisfactorily-functioning MRJAR/Java 9 module compilation solution in two/three weeks, then Gradle core devs could probably put my solution to shame in just a few hours.

So I’m not convinced that its purely technical reasons why Gradle doesn’t support MRJARs/Java 9 modules out the box.

But hey! I’m glad they don’t. Honest. Because the absense of that feature in Gradle, gives me something to tinker with in my spare time. So it’s all to the good :slight_smile:

1 Like

@tlinkowski: thanks for the info.

@tlinkowski: FYI: I was warned against ServiceLoader by @st_oehme, who mentioned that it is slow (I can’t remember if he had other objections). For my use case (which was multiple implementations of my own TaskConfigurer<T extends Task> interface, where T could be from any Gradle plugin, not just Gradle core), I just used the following method, which could be called if a supported Task implementation was detected (via detection of the application of the task type’s plugin to this project):

public void register(final TaskConfigurer<? extends Task> taskConfigurer) {
    taskConfigurerSet.add(taskConfigurer);
}
1 Like

@lingocoder

  1. I was able to create a MRJAR using v0.0.2.

  2. However, if I use the Application plugin, I get:

    FAILURE: Build failed with an exception.
    
    * What went wrong:
    A problem occurred configuring root project 'mrjartest'.
    > No value has been specified for this provider.
  1. The error message displayed if the JAVA_8 and JAVA_9 environment variables are not set is:
    You need to set two environment variables. One named '{}' and the other named '{}'. They must refer to the file system installations of JDK version {} and {} respectively.
  1. @rgoldberg made several great suggestions. I strongly encourage you to follow them.
1 Like

Awesome! Thanks again @siordache :+1:

Yeah. I think that’s related to this ticket I raised the other day.

Can you tell me, if not the precise steps I can repeat to reproduce what you got, then at least the first two or three steps? Please @siordache? TIA :+1:

Those placeholders again :anguished: I’ll get right on that. Thanks. I can’t tell you how much I appreciate your feedback. All of you all’s. What a super community :slight_smile:

Ah! I just remembered a feature of mrJar that I forgot to document anywhere. So sorry :blush:

As that „flaky main“ ticket I raised the other day explains, mrJar wires itself up to the core Application plugin’s :run task (@sterling clued me up to that neat trick).

So if you use Gradle’s Application plugin with mrJar in the mix, then, as the Steps to reproduce section of Issue #10412 explains, you need to set another one of mrJar’s optional properties:

...
mrJar {
    releases [...]
    main = 'com.acme.main.JavaMain'
}
...

Try adding that main property to your project? Please? My hypothesis for the root cause of #10412 is there might be something buggy going on in the innards of TestKit with regards to propagating dynamic properties. Or something. So it’d be valuable to learn what happens when the mrJar’s main property is set and read in a context other than TestKit.

Thanks again for reminding me to add another documentation task to my TODO list, @siordache :+1: