Incremental build fails - cannot find symbol

Help/Discuss
1

Hi,

Im currently seeing a problem with incremental compilation, where I get a ‘cannot find symbol’ when compiling.

This seems to be due to the way we generate some of our code, where our models inherit eachother and the resulting generated code will have the same package and class name but exist in the different subprojects (classpath overloading). The classes are not inherited.

This is all fine when we package the softwhere, we can control the classpath.

However, sometimes when a incremental compile is needed the ‘cannot find symbol’ error occurs. I’ve looked a bit into it and it seems that the output of the previous build is added to the compile classpath, but at the end of it, hence the wrong class is loaded first (the least specialized one).

If the output of the previous build is added first to the compile classpath the compilation is successful, I’ve tried to do this here: dvaske@296edb1. But I’m a bit unsure if this is the right path.

I’ll be happy to provide a pull request for this, if it seems good.
Also raised as a issue on github: https://github.com/gradle/gradle/issues/10606

Any thoughts?

-Aske

2

It’s not clear exactly what you’re doing from your description. Can you put together a minimal sample project which exhibits the problem? You might even spot the issue as you are creating the sample project.

3

Hi,
I’ve put a minimal example here:

To reproduce first run gradlew build then change CompanyImpl.java in the :a project to trigger recompilation (e.g. comment/uncomment the onmethod)
Now run gradlew build again and the compilation should fail in :b due to not being able to find PostingType._a_0

If you run with --debug you’ll see the Compiler args are (i’ve inserted newlines for better readability):

Compiler arguments: -source 1.8 -target 1.8 
-d /path/to/gradle-incremental-classpath/b/build/classes/java/main 
-encoding UTF-8 
-g 
-sourcepath  
-proc:none -s /path/to/gradle-incremental-classpath/b/build/generated/sources/annotationProcessor/java/main 
-XDuseUnsharedTable=true 
-classpath /path/to/gradle-incremental-classpath/a/build/classes/java/main:/path/to/gradle-incremental-classpath/b/build/classes/java/main 
/path/to/gradle-incremental-classpath/b/src/main/java/com/company/Company.java

As you can see the previous output of the :b project is last in the classpath, hence the PostyingType from :a is used in the recompilation, which then fails.

I hope this explains the issue a bit better.

4

Personally I think you should prevent duplicates rather than rely on classpath ordering. You could add a task to project B which takes a copy of project A’s classes filtering out the duplicates. You could then depend on the filtered copy rather than depending on project A

5

I see your point and agree to some extend. The duplicated classes exist in code genenerated from our model-tool. The models can extend each other, and in this scenario we get the duplicated classes.
I’ll try to see if I can figure out to do a classpath filter.
Any hints worth mentioning?

Thanks.

6

You could do something like this in project B.

configurations {
   temp { transitive = false } 
}
dependencies {
   temp project(':a')
   api files({tasks.filteredAJar})
}
task filteredAJar(type:Jar) {
   from configurations.temp.collect { it.directory?it:zipTree(it)}
   exclude { FileTreeElement el ->
      if (el.path.endsWith('.class')) {
         String javaPath = "src/codegen/java/${el.path.minus('.class')}.java"
         return file(javaPath).exists()
      } 
      return false
   } 
}
7

Cool, thanks, I’ll try this in the main project.

8

Obviously this is a hack, and the proper solution is to extract common classes to a shared module rather than generating the same classes twice

9

I tried a few different things yesterday.
It’s hard to extract the common classes to a shared module, as these comes from generated code. It might be possible to do something there though.

The filtering didn’t really work out in the main project, but I tried another “hack”:
adding

dependencies {
	implementation files('build/classes/java/main') // Load previous output first on compile classpath if any output
}

To all subprojects seems to do the trick.

10

Are the generated files exactly the same in project A and project B? Perhaps you could remove/exclude the duplicate generated java in project B from compilation. Or if the generated files are based on inputs (xml etc) you could exclude the inputs from the generate task in project B so the duplicates never exist in the first place

11

The generated files are not the same in A and B, they can be, but they can also be different…
And in the real world project we have a lot of projects where we have lots of layers of project dependencies. I’ll try to work on a filter or maybe exclude generated files from the dependency if the project is generating its own files.

12

Perhaps a simpler solution is this: Instead of ProjectB depending on the compiled classes from ProjectA, you could add any generated java files from ProjectA to the ProjectB compile task (which haven’t been overridden in ProjectB)

In this scenario there will be some java files in ProjectA which are compiled twice. And ProjectA will not be on ProjectB’s classpath

13

I ended up with separating the generated sources from the main sourceSet. So I would first generate the codegen sourceSets sources, then compile them and zip them to a Jar. The main sources are then made dependent on this jar.
This seems to work fine and is implemented as a plugin in buildSrc. One thing however is in the build.gradle files where I declare my dependencies

dependencies {
    api project(':a')

    // I now also need this dependency for the compileclasspath of the codegen sourceset
    codegen project(path: ':a', configuration: 'codegen')
}

Could I somehow extract that information in the buildSrc plugin, to be dependent on the same projects as api? So I would only need the first line?

I’ve tried to set the compileclasspath to only the projects in the main compileclasspath, but is do not seem to work:

project.sourceSets.codegen{
    classpath = project.configurations.compileClasspath.filter { !it.endsWith('.jar') }
}

But then I would get a

FAILURE: Build failed with an exception.

* What went wrong:
Circular dependency between the following tasks:
:b:codegenClasses
\--- :b:compileCodegenJava
     \--- :b:codegenJar
          \--- :b:codegenClasses (*)

(*) - details omitted (listed previously)
14

I put together a proof of concept here showing how I’d do it by splitting into four projects

  • a
  • a-codegen (depends on a, generates A.java,B.java,C.java & D.java)
  • b (depends on a)
  • b-codegen (depends on b, generates C.java & D.java)

You can see here where I have an “extendsFrom” concept

	codegen {
		extendsFrom project(':a-codegen')
		packageName = 'com.foo'
		classNames = ['C','D']
	}

ExtendsFrom logic here and here

See the test case here which verifies that the classpath is as expected.