Sorting .java files would help ensure more stable .class files

Help/Discuss
1

We just discovered when we started using lambda expressions in JDK 8 that .class files can vary from compile to compile, particularly from machine to machine where the environments may differ. We haven’t seen this before. We’ve come to expect that .class files don’t change unless the java source changes, however the java compiler offers no such contract in general. It only places a guarantee on runtime behavior, and makes no commitment to create identical byte code for any given source. In particular, when java source files are compiled (via the java-compiler-args.txt in the case of gradle) with the source files listed in a different order, you may very well end up with synthetic methods and attributes in a different order with different numeric suffixes.

Although this bug, https://bugs.openjdk.java.net/browse/JDK-8067422, fixed in JDK9 (but not currently in JDK8) mentions the case above, this is but one example. It would be good if the gradle java compile would sort the source files to reduce (but will never be able to totally eliminate) the problem.

I patched my copy of gradle 2.5 and verified that this leads to stability in the class files using jDK8. The patch I made is to org.gradle.api.internal.tasks.compile.JavaCompilerArgumentsBuilder:

private void addSourceFiles() {
    if (!includeSourceFiles) {
        return;
    }

    // putting the .java files in sorted order helps ensure that the java compiler doesn't
    // introduce changes in repeated compiles .class files
    // jim.gish@oracle.com - Aug. 27, 2015
    List<File> sortedFiles = new ArrayList<File>(spec.getSource().getFiles());
    Collections.sort(sortedFiles);

    for (File file : sortedFiles) {
        args.add(file.getPath());
    }
}

Please make this or equivalent change to the next version of gradle possible.

Thanks,
Jim

2

Hey @Jim_Gish,

Does this cause a problem?

It seems like it could break incremental compile where the generated class files aren’t cleaned up properly when the lambda is renumbered. Were you seeing this or did you just see “weirdness” with class files changing that shouldn’t?

Thanks!

3

It was causing a problem for a downstream process that was used to construct minimal patches based on changes seen in jar files. It can also cause problems in unnecessary spinning of jar files published to our repositories where the publish is only done if a jar file changes (based on a jar diff of the contents). Without the sort, what I found is that it tends to be stable on repeated builds on the same machine, but different between machines.

I haven’t thought about incremental compiles.

All in all this would be a desirable change and we’d appreciate seeing it happen if possible. Thanks.

4

Thinking about incremental compiles – I believe everything is ok in the scenario you mention, because the synthetic methods are private, it doesn’t matter what they are called locally. It’s only that the compiler deals with a group of .java files being compiled together when generating the names and dumping the byte code. If you later only do some of them, you’re ok because the old class files left around are still valid.