We have a project (entity classes for use with Hibernate) that requires post-compilation byte-enhancement step.
If interested (skip if not): See Bytecode Enhancement. This modifies *.class files produced by the Java compiler. In pure Gradle build we do this by applying the ‘org.hibernate.orm’ plugin.
We had this working with Gradle 5.6.4 and older Eclipse by doing something that amounts to unpleasant hacking (will describe that below). That hack no longer works with Gradle 6.1.1 and Eclipse 2019-12 - not sure of the exact cause but the symptom is clear. I’d like to take the opportunity to do this right … but I don’t know how to.
As it is, Buildship imports this project as normal and leaves Eclipse’s Java builder in (after Gradle builder). Eclipse ends up not configured to know anything about bytecode enhancement. The result is that the code compiles with no errors but the product does not work as the code isn’t enhanced (required).
Note that we keep Eclipse output folders (bin/…) separate from Gradle’s (build/…). Sharing them does not work us (does it for anyone) due to different compiler/compilation and bytecode styles.
Some people who use IntelliJ are not reporting issues. My guess is that IntelliJ gives up on compiling the project itself and relies entirely on Gradle to do so (but IntelliJ has other issues for us). How do we set our project up properly so that it “just works”?
Please help!
Our past hack, for those interested, others please DO NOT READ
You’ve been warned. This is downright nasty. We had to:
Disable Eclipse’s own Java builder (for some/unknown reason Eclipse-produced bytecode is not enhanceable) on this project by creating a .externalToolBuilders/org.eclipse.jdt.core.javabuilder.launch file with the following content:
Create an ANT (!) build file that can launch Gradle for us so we can use Eclipse’s ant builders.
<?xml version = "1.0"?>
<project name="gradle-runner-ant" default="run-gradle" basedir=".">
<description>
Runs a Gradle task specified via the GRADLE_TASK system property.
This allows Eclipse Ant builders to be conveniently used to run Gradle tasks.
</description>
<target name = "run-gradle">
<echo message="Running Gradle: ${GRADLE_TASK}"/>
<java fork="true" failonerror="yes" classname="org.gradle.wrapper.GradleWrapperMain" classpath="gradle/wrapper/gradle-wrapper.jar">
<arg line="${GRADLE_TASK}"/>
</java>
</target>
</project>
Register that external tool builder in our *.gradle file:
eclipse {
project {
String buildCommandName = 'org.eclipse.ui.externaltools.ExternalToolBuilder';
String launchHandle = '<project>/.externalToolBuilders/Gradle Java compiler with Hibernate bytecode enhancement.launch';
buildCommand buildCommandName, LaunchConfigHandle: launchHandle
file {
whenMerged { p ->
def commands = p.buildCommands;
// For some reason Eclipse-produced Java bytecode cannot be enhanced.
// We have to use our official compiler to yield correct results.
commands.removeAll { it.name == 'org.eclipse.jdt.core.javabuilder' }
int index = commands.findIndexOf {
(it.name == buildCommandName) && (it.arguments.LaunchConfigHandle == launchHandle)
}
def wsCommand = commands.remove(index);
commands.add(0, wsCommand);
// Do we have more than one builder of ours? It has been observed!
// Remove duplicates.
int duplicateIndex;
while ((duplicateIndex = commands.lastIndexOf { (it.name == buildCommandName) && (it.arguments.LaunchConfigHandle == launchHandle) }) > 0) {
commands.remove(duplicateIndex);
}
}
}
}
}
Create our own compileForEclipse task in Gradle that depends on Gradle’s standard “compileJava”, then copies the results into Eclipse’s output folders.
Finding: the code IS getting enhanced but some Java builder re-runs and overwrites the enhancements with original code… without re-running the enhancement step again. I’ve confirmed this. Tried various ways to convince Eclipse+Buildship to do this right but couldn’t.
Can anyone help?
As there was no traction here I posted a related question (from Eclipse perspective):
Not really our choice. This is not our enhancer (it is Hibernate) and the alternatives (runtime bytecode enhancement, use of reflection and/or forcing non-private visibility and having runtime wrappers with subpar functionality) are not an option for various reasons including runtime performance and interaction with unit tests.
External tool, operates on bytecode, not source. Even if it were the source code this would cause line numbers to not match in debugging and would cause hell that way.
That only adds complexity, moves the problem to another project, if we declare it a non-Java project it hides the dependency from Eclipse (for debugging purposes) and adds non-Eclipse complexity to Gradle as well.
Here’s a minimum example to reproduce (using actual bytecode enhancement) and with additional Eclipse Ant builders that simulate bytecode enhancement builder in there + some more, for troubleshooting. The zip file includes the output and the log file (see ant-builder-*.log files):
One interesting thing to note is that timestamps are not in order. Not sure why. That may be a clue.
UPDATE - I have this mostly figured out now. Will provide details as soon as I clean this up. I don’t have a definitive answer as to why the entries are not chronological, only a hunch, though.
There were a few things I was unclear/wrong about and couldn’t find relevant documentation to learn the details. Here’s a summary of what needs to be known to get this right (some of which I got right from the get go, but not everything):
Gradle/Buildship integration in Eclipse attempts to make use of Eclipse’s internal compiler. This is by Eclipse’s design and has its own advantages during development … as well as disadvantages during times like this - inability to leverage the external/production builders (Gradle model in this case) to do the building. For this reason any bytecode enhancement (plugins or not) operating inside Gradle will have no effect in Eclipse at all (unless they do something Eclipse-specific). (I got this right)
To perform bytecode enhancement in Eclipse (not apt or what some Eclipse plugin could do), one has to add custom builders and position them after the default Java builder (manually or automatically). (I got this right too)
Out of box Eclipse offers two kinds of builders - “Ant” and “Program”. No Gradle there. “Program” kind is not cross-platform, only Ant is. Ant can be used to do things or launch Gradle in a cross-platform way (Java exec on Gradle’s main() method). (I got this right too)
[I WAS WRONG HERE] Eclipse offers four “events” to which one can bind an Ant builder: After a “Clean”, Manual Build, Auto Build and During a “Clean”. I did not understand when do they run. For example, I thought that “During a Clean” runs while the clean is happening and that After a “Clean” is there to run after that, to allow the custom builder to do its own after-clean cleaning. I thought that this would be followed by Auto build if “Build automatically” is enabled or “Build immediately” is checked in the “Clean” dialog. This is NOT the case. After a “Clean” actually refers to a build step, not cleaning and will NOT be followed by the *Auto Build" step (that step only runs when one saves an edit and “Build Automatically” is enabled. In the builder’s *.launch file this is much more apparent - the After a “Clean” is actually called full and Auto and Manual builds are called auto and incremental. This means that en enhancement builder has to be set to run on After a “Clean” in addition to Auto Build and Manual Build and should NOT be set to run on *During a “Clean”. My mistake was to only set it to run for Auto and Manual builds.
[I WAS WRONG HERE TOO] I specified the working set of relevant resources in the Build Options tab for the enhancement builder. I set those resources to be the source code (to be enhanced) and build.gradle (containing the enhancer). Since these do not change on most builds, Eclipse chooses not to run the builder. The now obvious 20-20 vision truth is that relevant resources for this builder are the Java builder’s output binaries (and build.gradle), not the Java source code. However, this isn’ the entirely correct choice (in isolation) either as Eclipse, in our case, ends up in an infinite loop - it thinks that the enhancer changed the binaries and, as it is set to run when binaries changed, runs the build again. We cannot NOT set the relevant resources at all as that seems to mean “everything/anything”. The enhancer must be made in such a way to NOT even touch the files that are already enhanced [UPDATE] and that isn’t enough. Read on.
I still don’t definitively know why the informational builders I used to research this have their output appended to the common log file in order that isn’t chronological. I can only assume that this has to do with Eclipse’s output buffering and periodic writing to these files somehow.
[UPDATE 1]
[I DIDN’T KNOW THIS] Eclipse has a workspace setting (checkbox, overridable per project) in Preferences -> Java -> Compiler -> Building -> Output Folder called “Rebuild class files modified by others” officially described as “Indicate whether class files which have been modified by others should be rebuilt to undo the modification.”. By default it is unchecked/off, which seems right for this case. This, however, does not work as advertised. Whatever the setting is, Eclipse’s Java Builder will react to its output changes (as if they were inputs, I call this a defect) and “undo the modification”, causing infinite build loops. I found this symptom reported many times - search for this. With “correct” setup and no hacking the Java builder keeps undoing the modifications and Eclipse keeps re-running them, causing the infinite loop regardless of the setting.
[HACK THAT SEEMS TO WORK PRESENTLY] In addition to having everything above set up correctly I modified the enhancer to ensure two things (either only one or both may be required, not sure): (a) that existing *.class files are NOT deleted and recreated but rewritten and (b) that their last modified time is changed back to what it was before the enhancement. This seems to trick the Eclipse’s modification detection enough to break out of the loop even though the file sizes are different. This is with Eclipse 2019-12 (4.14.0.v20191210-0610) and it may stop working with any update. I hope they fix the infinite build loop defect by then.
[UPDATE 1] This file/example does not have the “UPDATE 1” changes above as they are not all that trivial for a succinct example. However, in all other respects it is correct.
Actually … this is not the ultimate answer yet… In a larger project this, depending on how “relevant resources” are set up either ends up in an infinite builder loop (when “Build automatically” is enabled) or ends up having the initial problem.
Added [UPDATE 1] to my answer above to keep the solution together - I addressed the challenge above.
Extra note to Gradle & Buildship developers: I haven’t ruled out that the above Eclipse misbehaviour is (not) specific to Gradle/Buildship projects. There remains a possibility that Buildship is the cause of this, however small.