I just ported one of our big project to Gradle and we just found out (via scans) that compileJava on simple modifications is taking much longer compared to our previous setup (plain project, dependencies included via Netbeans Libraries).
Previously, whenever something was modified only a few classes had to be recompiled (modifying a String triggers just 8 class for recompilation, to say), and the whole takes ~1s.
Now with Gradle we sit on 13-16s
Gradle 5.5.1
Any hints?
Share the build scan.
Or read through the build with --info to see why the unexpected tasks are being triggered.
Looks like everything is a large module.
Step 1:
That will get you most of the improvements you want without much hassle.
Step 2:
Then when you’re confident the incremental support is working, enable local build caches.
Step 3:
When you’re confident local caching is working, enable a remote build cache populated from CI.
Hi Giuseppe,
did you check out if incremental Java compilation is enabled: https://docs.gradle.org/current/userguide/java_plugin.html#sec:incremental_compile ?
It should be enabled by default. You can run with --info to see how many classes are recompiled. Normally, only a fraction of the classes should be recompiled.
Cheers,
Stefan
Honestly I’m reluctant to do this because this means some overhead like restructuring the project into modules (having different sources under different module/src/main/java) and with their corresponding build.gradles and their relative inter-dependencies
I’ll look into 2, for 3 I have no idea what it means, I’ll reserach
Hi Stefan,
how can I check it? --info clearly mention full recompilation
Hi Giuseppe,
does it also mention a reason why full recompilation happens? When it shows full recompilation, then it means that all classes are re-compiled. And that explains the extra time the task takes.
Cheers,
Stefan
You can see in the link above (here again), it’s a gist (I just modify a System.println in the mentioned class to simulate the smallest change):
Task ‘:compileJava’ is not up-to-date because:
Input property ‘source’ file D:\DEVELOPMENT\EMM-Check\src\main\java\ec\gui\dialogs\vrserver\EC_VrClientListener.java has changed.
Created classpath snapshot for incremental compilation in 0.005 secs. 2061 duplicate classes found in classpath (see all with --debug).
Class dependency analysis for incremental compilation took 0.075 secs.
Full recompilation is required because ‘EC_VrClientListener.java’ was changed. Analysis took 0.083 secs.
Compiling with JDK Java compiler API.
Btw, this prints true, so I guess incremental compilation is on
tasks.withType(JavaCompile) {
println("options.incremental: $options.incremental")
}
Are you changing a constant in the class? This is from the user manual:
Since constants can be inlined, any change to a constant will result in Gradle recompiling all source files. For that reason, you should try to minimize the use of constants in your source code and replace them with static methods where possible.
No, just a println in a constructor
public EC_VrClientListener(REKNOW_Client client) {
this.client = client;
startPolling();
System.out.println("asa");
}
IIUC, if the class has public constants, then it is considered a dependency to all other classes. See https://github.com/gradle/gradle/issues/6482.
I also found continuous build to be quite useful… if I could only set that from build.gradle instead of console, it’d be perfect…
Anyway, putting aside for a moment tricks to speed up Gradle, the question remains: why a plain project without build tool is much faster?
I’m inclined to think is because of IDE optimizations… in Idea-Gradle setting I read indeed:
In a pure Java/Kotlin project, building and running by means of the IDE might be faster, thanks to optimizations
This question comes up all the time. I also have the impression (anecdotal evidence - therefore not worth much) that Gradle is slower then for example Ant.
Maybe the Gradle team could make a poll where people can suggest 3 big and popular open source Java projects. Once 3 projects are selected the Gradle team writes builds for all of them in Gradle and shows through some build time comparison that Gradle is in fact not slower. Also it could show how easy it is to move to Gradle with your build.
Then host those comparisons (including all necessary files to run them) on Github and people can benchmark themselves.
This build performance comparison Github project could serve as a future test to see if build time went down or got faster with future releases of Gradle.
Fun fact: Netbeans uses actually Ant for building and other stuff…
True. And it is slower then IDEA. And IDEA is slower in building the Eclipse. Gradle should take a hard look at what Eclipse is doing and copying it. Eclipse builds a model from source code so it knows about Global Constants and inlining is no problem. Full recompiles are NEVER necessary.
Hi Dieter,
We (the Gradle team) has quite an extensive suite of performance tests making sure Gradle is not getting slower and that it stays faster than e.g. Maven. See for example https://gradle.org/gradle-vs-maven-performance/.
Normally, Gradle will be much faster that Ant and Maven for incremental scenarios, since neither Ant nor Maven support incremental builds nor incremental compilation.
If it is not faster, then something is wrong which should be fixed. The first place to look how to make your builds faster is the performance guide. If that does not help, ask in the forums (here) or open an issue.
The issue we are discussing here is https://github.com/gradle/gradle/issues/6482, We currently don’t have the bandwidth to address it, and there is a workaround. Upvoting the issue makes it more likely to be implemented. Currently, it has 0 upvotes.
Cheers,
Stefan
Hi Stefan,
I agree that bandwidth is key here. Still there are many possible solutions and some have basically zero bandwidth costs:
Maybe there is an unpublished flag for javac that prevents inlining of public constants and you allow people to turn it on? Maybe reach out to people at Oracle (if the flag does not exist convince them to add it) or dig through the OpenJDK sources to find the flag.
Could the annotation mechanism somehow prevent constant inlining? People could then add those annotations to their constants.
Can’t you just check if .java files containing public constants have changed since the last compile to prevent a full recompile? I know this doesn’t cover all cases of full recompile but it would be a start. Or are you doing this already?
Use spoon or Eclipse JDT to build a source model that knows when constants have changed. This would basically make full recompiles a thing of the past. Arguably this is also the most costly in terms of bandwidth to implement.
P.S.: I still think a single Github project that builds popular projects with Gradle and compare that to their original build mechanism (Maven, ANT, etc.) in terms of performance would be of great benefit to Gradle.
1.: There is no flag for javac.
2: Why don’t you replace the constants by getters? That solves the problem - no more inlining.
3: We already do this: if the changed Java files do not contain any constants, only the required files are recompiled and not everything
4: That is very involved and we would move away from using javac which doesn’t sound like the direction we want to go.
If you have a Github project at hand, then we are more than happy to compare the different build systems and different Gradle versions there. We currently create our own test projects, feel free to check them out. The instructions how to create them are here.