We have a legacy application and legacy build tool we are using to build both our native and Java code. For Java, we tied “JMake” into our build tool, however, it is no longer maintained, does not support JDK 9 and above, and we’re needing to move to Java 9 / 10 / 11.
In our environment, we have “baseline” builds (with full source code) and the developers use “sparse work areas” (only the code they are changing).
I’ve figured out how to get Gradle to find the source files (by giving the required srcDirs and using include “seta/setb/foo/**/*.java”) and that works okay, but when I give both the baseline src area and the developer src are, I get complaints about classes being duplicated.
So, it seems to me that I need to write a plugin that adapts the “java” plugin so that it understand our model and only takes the first java file with a given name that it finds. In other words, when given these areas to search for source files, with “MyFile.java” in both, only the developer’s file should be used:
/developer/src/com/slb/test/MyFile.java <== should be used
/baseline/build-01/src/com/slb/test/MyFile.java <== should be ignored
[it is possible to have multiple layers, all of which may have a MyFile.java; in all cases only the “first” should be used. e.g. /developer/src, /baseline/build-03, /baseline/build-02, /baseline/build-01 – effectively stacked incremental build areas]
am I correct that I need a special plugin for this because I cannot just give the java plugin a list of files to build?
if so, can I extend the “java” plugin?
… or must I duplicate all the code from it and then make the changes I need? [I really don’t want to do that]
Alternatively, we already have an Ant-based class that understands the above “search path” model (called GFileSet based on Ant’s FileSet), but I don’t see how I can get this to the java plugin so it will be used instead of the conventional “include **/*.java”.
I assume that the “sparse work areas” are to reduce compilation times. You’ll likely find that Gradle’s change detection and dependency tracking eliminate this need.
Just try to compile all the sources, change a single file and recompile. If the time is unacceptable, do not hesitate to post to this forum.
One more thing, unlike Ant and Maven, Gradle tries hard to figure inter-class dependencies and recompile the impacted files (not sure if it works for primitive constants changing), but in general I have rarely seen a case when I have needed to do a clean build.
Yes, that’s part of it, but the issue is we have tons of code (10GB in our src area) and so having a complete copy of “src” locally is not feasible in our environment. Our legacy build tool swallows all of that and builds it all, in order, in one pass.
Our Java code is interspersed with C, C++, Fortran, Fortran90, Perl, and Python among others which I’m probably forgetting.
Replacing our legacy build tool with Gradle isn’t an option right now, and may never be. What I was looking to do was use Gradle to handle Java (due to the points you made about it) and leave the rest to our existing tools…
Our existing tool can be made to handle Java, but it is a very dumb “oh, you changed Foo.java, so I will rebuild it as Foo.class” but does not take into account the need to rebuild dependents, too – which is why we were using Jmake instead.
I don’t like this idea, but I guess I could rsync the required code – after doing the filtering we require – into “src/main/java” and then just use the defaults; surely there’s a better way though…
It looks like “Plugin Composition” may allow me to customize the “java” plugin to do what I need it to do. Still investigating, but that seems more promising than maintaining my own fork of the java plugin.
Since the java plugin is driven by SourceSet and SourceDirectorySet I think you’ll manually configure the java tasks rather than using the java plugin. Possibly something like
See Lance’s reply for how to customize the compile task.
If you have many filesets, you may want to express them declaratively in some map structure and use Task Rules to create the compile tasks on demand.
This way, adding a new “working set” becomes as simple as adding another key/value in the map, which would probably live in its own build file or be loaded from some data (json, csv, or groovy config slurper).
I started a new project area, ran “gradle --init” and updated build.gradle with contents as above, changing “src/main/…” entries to match our source code structure.
That changes removes the complaint, but now yields:
Caused by: java.lang.NullPointerException
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ‘:compileJava’.
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:49)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
perhaps either source isn’t getting set, or one [or all 3?] of my “tree” objects have no content…?
I get similar NPE from using source = tree3
EDIT: For what it’s worth, “tree3” isn’t empty; I did an each / println on it.
Perhaps try a small standalone project first with src/level1/java and src/level2/java etc with a couple of java files to test the override functionality. If you commit it to github I might even take a look if you can’t get it working.
Also try adding --stacktrace to the command line to see the full trace.
C:\path>gradle printTrees
> Task :makeFiles
Created C:\path\src\level1\java\com\foo\A.java
Created C:\path\src\level1\java\com\foo\B.java
Created C:\path\src\level1\java\com\foo\C.java
Created C:\path\src\level1\java\com\foo\D.java
Created C:\path\src\level1\java\com\foo\E.java
Created C:\path\src\level1\java\com\foo\L1.java
Created C:\path\src\level2\java\com\foo\A.java
Created C:\path\src\level2\java\com\foo\B.java
Created C:\path\src\level2\java\com\foo\L2.java
Created C:\path\src\level3\java\com\foo\A.java
Created C:\path\src\level3\java\com\foo\L3.java
> Task :printTrees
C:\path\src\level3\java\com\foo\A.java
C:\path\src\level3\java\com\foo\L3.java
C:\path\src\level2\java\com\foo\B.java
C:\path\src\level2\java\com\foo\L2.java
C:\path\src\level1\java\com\foo\C.java
C:\path\src\level1\java\com\foo\D.java
C:\path\src\level1\java\com\foo\E.java
C:\path\src\level1\java\com\foo\L1.java
As far as I can see, the stack trace is saying that “source” is empty.
What went wrong:
Execution failed for task ‘:compileJava’.
java.lang.NullPointerException (no error message)
Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ‘:compileJava’.
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:49)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$ExecuteTaskAction.execute(DefaultTaskExecutionGraph.java:262)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$ExecuteTaskAction.execute(DefaultTaskExecutionGraph.java:246)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:136)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.execute(DefaultTaskPlanExecutor.java:201)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.executeWithTask(DefaultTaskPlanExecutor.java:192)
at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: java.lang.NullPointerException
at org.gradle.jvm.platform.internal.DefaultJavaPlatform.generateName(DefaultJavaPlatform.java:63)
at org.gradle.jvm.platform.internal.DefaultJavaPlatform.(DefaultJavaPlatform.java:30)
at org.gradle.api.tasks.compile.JavaCompile.getPlatform(JavaCompile.java:157)
at org.gradle.api.tasks.compile.JavaCompile_Decorated.getPlatform(Unknown Source)
at org.gradle.api.internal.tasks.properties.bean.AbstractNestedRuntimeBeanNode$DefaultPropertyValue$1$1.create(AbstractNestedRuntimeBeanNode.java:83)
at org.gradle.util.SingleMessageLogger.whileDisabled(SingleMessageLogger.java:200)
at org.gradle.api.internal.tasks.properties.bean.AbstractNestedRuntimeBeanNode$DefaultPropertyValue$1.get(AbstractNestedRuntimeBeanNode.java:80)
at com.google.common.base.Suppliers$MemoizingSupplier.get(Suppliers.java:125)
at org.gradle.api.internal.tasks.properties.bean.AbstractNestedRuntimeBeanNode$DefaultPropertyValue.getValue(AbstractNestedRuntimeBeanNode.java:138)
at org.gradle.api.internal.tasks.properties.annotations.NestedBeanAnnotationHandler.visitPropertyValue(NestedBeanAnnotationHandler.java:46)
at org.gradle.api.internal.tasks.properties.bean.AbstractNestedRuntimeBeanNode.visitProperties(AbstractNestedRuntimeBeanNode.java:62)
at org.gradle.api.internal.tasks.properties.bean.RootRuntimeBeanNode.visitNode(RootRuntimeBeanNode.java:32)
at org.gradle.api.internal.tasks.properties.DefaultPropertyWalker.visitProperties(DefaultPropertyWalker.java:41)
at org.gradle.api.internal.tasks.TaskPropertyUtils.visitProperties(TaskPropertyUtils.java:39)
at org.gradle.api.internal.tasks.execution.DefaultTaskProperties.resolve(DefaultTaskProperties.java:77)
at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:53)
… 21 more
After much trial-and-error, I am able to get a uniq list of java files and pass them to the JavaCompile class.
The next hurdle is how to turn this into a jar file. I get a jar file, but it has only the manifest in it.
I’ve not yet figured out the right things to enter for the jar section:
version = “0.0.1”
task myJar (type: Jar) {
manifest {
attributes(‘Implementation-Title’: project.name,
‘Implementation-Version’: project.version)
}
from sourceSets.main.output
include “**/*.class”
}