Task ':app:compileDebugAidl' class path has changed

fab1an · September 19, 2017, 7:18am

I have configured Android Studio to use the same gradle-daemon that I use on the command-line: https://medium.com/@spitzwegerich/easily-improved-gradle-android-studio-performance-700b84e43602

Building and rebuilding on the command-line works and caches correctly
Building and rebuilding in Android Studio works and caches correctly.

When I switch between the two, though, executing the exact same gradle task in Android Studio rebuilds every time. The given reason:

Executing task ':app:compileDebugAidl' (up-to-date check took 0.005 secs) due to:
  Task ':app:compileDebugAidl' class path has changed from 74ca6ed2c00f9c841864b3c55c165b01 to 47615d314618a828970b7a001297b834.
All input files are considered out-of-date for incremental task ':app:compileDebugAidl'.

Is there any way to debug this? I’ve spend three hours here already :(.

Stefan_Wolf · September 19, 2017, 7:42am

Hi Fabian,

Which Gradle version are you using? Are you sure you are running the build with the same daemon? The message you are showing points to some changes to the classpath of the compileDebugAidl task. I guess this task is coming from the Android plugin. The classpath of the task contains

the android plugin jar itself and all its dependencies
the buildSrc jar
the Gradle version
the Java version

Can you make sure that these do not change between invocations?

Cheers,
Stefan

fab1an · September 19, 2017, 9:12am

I made sure that it’s the same daemon.

Unfortunately I don’t kow how to make sure the other things stay the same.
Is there a way of showing those in the output?

android-plugin-jar: should be the same, since it’s taken from gradle?
buildSrc: I don’t touch the buildSrc in-between.
Gradle version: Same daemon = same gradle-version, am I correct?
Java version: Android Studio usually starts a second daemon because it uses it’s own jdk, hence JAVA_HOME differs. I configured it so that it uses my system-jdk for gradle, so it doesn’t start a second daemon anymore.

Build output from AndroidStudio after running clean, then compileAidl twice:

Executing tasks: [compileDebugAidl]

:buildSrc:compileJava NO-SOURCE
:buildSrc:compileGroovy UP-TO-DATE
:buildSrc:processResources NO-SOURCE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar UP-TO-DATE
:buildSrc:assemble UP-TO-DATE
:buildSrc:compileTestJava NO-SOURCE
:buildSrc:compileTestGroovy NO-SOURCE
:buildSrc:processTestResources NO-SOURCE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test NO-SOURCE
:buildSrc:check UP-TO-DATE
:buildSrc:build UP-TO-DATE
NDK is missing a "platforms" directory.
If you are using NDK, verify the ndk.dir is set to a valid NDK directory.  It is currently set to /Users/fabian/Library/Android/sdk/ndk-bundle.
If you are not using NDK, unset the NDK variable from ANDROID_NDK_HOME or local.properties to remove this warning.

The setTestClassesDir(File) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use the setTestClassesDirs(FileCollection) method instead.
The getTestClassesDir() method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use the getTestClassesDirs() method instead.
The ConfigurableReport.setDestination(Object) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use the method ConfigurableReport.setDestination(File) instead.
:app:generateJSX UP-TO-DATE
:app:preBuild UP-TO-DATE
:app:extractProguardFiles
:app:preDebugBuild
:app:checkDebugManifest
:app:preReleaseBuild
:app:prepareComAndroidSupportAnimatedVectorDrawable2340Library
:app:prepareComAndroidSupportAppcompatV72340Library
:app:prepareComAndroidSupportDesign2340Library
:app:prepareComAndroidSupportRecyclerviewV72340Library
:app:prepareComAndroidSupportSupportV42340Library
:app:prepareComAndroidSupportSupportVectorDrawable2340Library
:app:prepareComFacebookFbuiTextlayoutbuilderTextlayoutbuilder100Library
:app:prepareComFacebookFrescoDrawee130Library
:app:prepareComFacebookFrescoFbcore130Library
:app:prepareComFacebookFrescoFresco130Library
:app:prepareComFacebookFrescoImagepipeline130Library
:app:prepareComFacebookFrescoImagepipelineBase130Library
:app:prepareComFacebookFrescoImagepipelineOkhttp3130Library
:app:prepareComFacebookReactReactNative0483Library
:app:prepareComFacebookSoloaderSoloader010Library
:app:prepareComJakewhartonThreetenabpThreetenabp105Library
:app:prepareComMapboxMapboxsdkMapboxAndroidSdk422Library
:app:prepareComMapzenAndroidLost111Library
:app:prepareOrgWebkitAndroidJscR174650Library
:app:prepareDebugDependencies
:app:compileDebugAidl UP-TO-DATE

BUILD SUCCESSFUL in 1s
24 actionable tasks: 22 executed, 2 up-to-date

Immediate run of ./gradlew compileAidl console=plain on the console

:buildSrc:compileJava NO-SOURCE
:buildSrc:compileGroovy UP-TO-DATE
:buildSrc:processResources NO-SOURCE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar UP-TO-DATE
:buildSrc:assemble UP-TO-DATE
:buildSrc:compileTestJava NO-SOURCE
:buildSrc:compileTestGroovy NO-SOURCE
:buildSrc:processTestResources NO-SOURCE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test NO-SOURCE
:buildSrc:check UP-TO-DATE
:buildSrc:build UP-TO-DATE
NDK is missing a "platforms" directory.
If you are using NDK, verify the ndk.dir is set to a valid NDK directory.  It is currently set to /Users/fabian/Library/Android/sdk/ndk-bundle.
If you are not using NDK, unset the NDK variable from ANDROID_NDK_HOME or local.properties to remove this warning.

The setTestClassesDir(File) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use the setTestClassesDirs(FileCollection) method instead.
The getTestClassesDir() method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use the getTestClassesDirs() method instead.
The ConfigurableReport.setDestination(Object) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use the method ConfigurableReport.setDestination(File) instead.
:app:generateJSX UP-TO-DATE
:app:preBuild UP-TO-DATE
:app:extractProguardFiles
:app:preDebugBuild
:app:checkDebugManifest
:app:preReleaseBuild
:app:prepareComAndroidSupportAnimatedVectorDrawable2340Library
:app:prepareComAndroidSupportAppcompatV72340Library
:app:prepareComAndroidSupportDesign2340Library
:app:prepareComAndroidSupportRecyclerviewV72340Library
:app:prepareComAndroidSupportSupportV42340Library
:app:prepareComAndroidSupportSupportVectorDrawable2340Library
:app:prepareComFacebookFbuiTextlayoutbuilderTextlayoutbuilder100Library
:app:prepareComFacebookFrescoDrawee130Library
:app:prepareComFacebookFrescoFbcore130Library
:app:prepareComFacebookFrescoFresco130Library
:app:prepareComFacebookFrescoImagepipeline130Library
:app:prepareComFacebookFrescoImagepipelineBase130Library
:app:prepareComFacebookFrescoImagepipelineOkhttp3130Library
:app:prepareComFacebookReactReactNative0483Library
:app:prepareComFacebookSoloaderSoloader010Library
:app:prepareComJakewhartonThreetenabpThreetenabp105Library
:app:prepareComMapboxMapboxsdkMapboxAndroidSdk422Library
:app:prepareComMapzenAndroidLost111Library
:app:prepareOrgWebkitAndroidJscR174650Library
:app:prepareDebugDependencies
:app:compileDebugAidl

BUILD SUCCESSFUL in 1s
24 actionable tasks: 23 executed, 1 up-to-date

I really don’t know how to proceed in debugging this.

Stefan_Wolf · September 19, 2017, 9:32am

Thank you for the added information.
Could you provide the --info log, too? If you don’t want to post it in the forum you can also send it to me (wolf at gradle dot com).
I guess when you run the task the second time in IDEA, then it is up-to-date, isn’t it?
What version of Android Studio and Gradle are you using? Could you share a project so we can reproduce the issue?

fab1an · September 19, 2017, 10:08am

Yes, the second time it is up-to-date, in IDEA as well as on console, it’s just when I switch, the information is lost.

I’m using Android Studio 2.3.3 and also com.android.tools.build:gradle:2.3.3.
I used gradle 4.1.

I sent you the logs via email.

Stefan_Wolf · September 20, 2017, 9:46am

I still don’t have enough information to get some ideas…
Could you add the following snippet to your build.gradle:

import org.gradle.internal.classloader.ClassLoaderVisitor
import org.gradle.internal.classloader.ClassLoaderHierarchyHasher

def hasher = gradle.services.get(ClassLoaderHierarchyHasher)

gradle.buildFinished {
  def task = project(':app').compileDebugAidl
  new Visitor(hasher, task).visit(task.getClass().getClassLoader())
}

class Visitor extends ClassLoaderVisitor {
    final ClassLoaderHierarchyHasher hasher
    final String prefix
    int counter

    Visitor(ClassLoaderHierarchyHasher hasher, Task task) {
        this(hasher, "${task.path}-classloader")
    }

    Visitor(ClassLoaderHierarchyHasher hasher, String prefix) {
        this.hasher = hasher
        this.prefix = prefix
    }

    private String classloaderHash(ClassLoader loader) {
        def hash = hasher.getClassLoaderHash(loader)
        if (hash != null) {
            return hash.toString()
        } else {
            null
        }
    }

    @Override
    void visit(ClassLoader classLoader) {
        def hash = classloaderHash(classLoader)
        if (hash) {
            def classloaderName = classLoader.getClass().getSimpleName()
            println "${prefix}-${counter++}-${ classloaderName}-hash:  $hash"
            if (classLoader instanceof URLClassLoader && (!classloaderName.contains('ExtClassLoader'))) {
                println "${prefix}-${counter - 1}-classpath ${classLoader.getURLs().join(':')}"
            }
        }
        super.visit(classLoader)
    }
}

This will print out the classloader hashes of the classloader of the class and its parents. If you could provide the output for running compileDebugAidl in IDEA and on the commandline I probably will have a better idea what is actually different. Either post the output here or send it to me via mail.

Thank you very much for your help in getting to the root of the issue!

fab1an · September 21, 2017, 3:10am

Sure.
Thank you for the help, I really appreciate it.
Fixing this will save me and potentially others many flow-breaking minutes of waiting per day.

Stefan_Wolf · September 21, 2017, 8:43am

Hi Fabian.

Thank you again for the logs you sent me.
After looking closer at the issue it seems that Android studio injects a custom file repository into the Gradle build. For some reason, when using the custom repository, antlr 3.5.2 is not added to the build script classpath, while it is added when running from the command line.
It seems that Android Studio 3.0 (beta 6 at the time of writing) should not do that any more. Could you verify that the task is up to date when using Android Studio 3.0?

Cheers,
Stefan

fab1an · September 22, 2017, 9:49am

Hi, Stefan

I just installed AS3-beta6 and sadly the issue isn’t fixed. I have generated the same logs again and sent them per email.

Stefan_Wolf · September 25, 2017, 1:55pm

Hi Fabian,

thanks again for the logs. It looks like nothing changed. Could you also try using the latest Android plugin for Gradle? Here are migration instructions: https://developer.android.com/studio/build/gradle-plugin-3-0-0-migration.html
What you also could try is to exclude antlr:3.5.2, since these are the offending artifacts which are present in the buildscript classpath when running from the console:

buildscript {
    configurations.classpath.exclude group: 'org.antlr', module: 'antlr'
    configurations.classpath.exclude group: 'org.antlr', module: 'antlr-runtime'
    configurations.classpath.exclude group: 'org.antlr', module: 'ST4'
}

Cheers,
Stefan

Stefan_Wolf · September 26, 2017, 7:18am

Hi Fabian,

on another note, it seems like the problem is not Gradle related at all, but more a problem with Android Studio/the Android plugin for Gradle. Could you open an issue for the Android plugin for Gradle?

Cheers,
Stefan

fab1an · September 26, 2017, 10:05am

Hi Stefan,

I’m using classpath 'com.android.tools.build:gradle:3.0.0-beta6'already, since the gradle-plugin version must always be tied to the Android-Studio version.

I’ve reported a bug: https://issuetracker.google.com/issues/66936393

I hope someone will look at it. These “minor” bugs tend to get ignored, but it is my opinion that improving build-performance and caching is a multiplicator that increases quality-of-software (and quality-of-life) everywhere.

In that sense thank you again for your time looking at this.

fab1an · September 26, 2017, 10:09am

Also I tried the tip with the exclusions, but no difference.

Stefan_Wolf · September 26, 2017, 11:12am

Hi Fabian,

I did something wrong when looking at the second set of logs with Android Studio 3.0 (I looked at the wrong log files). Actually, it seems that all the jars are the same (same name, same version). It could be possible that one of the jars has a different content. Do you use any custom repositories in your build script? I could send you a script to get the md5 sums of all the libraries involved and then we could compare those. Would you be willing to try that out?

Cheers,
Stefan

Stefan_Wolf · September 26, 2017, 11:21am

This would be the code you would need to add to your build script:

import org.gradle.internal.classloader.ClassLoaderVisitor
import org.gradle.internal.classloader.ClassLoaderHierarchyHasher
import org.gradle.internal.hash.FileHasher

def hasher = gradle.services.get(ClassLoaderHierarchyHasher)
def fileHasher = gradle.services.get(FileHasher)

gradle.buildFinished {
  def task = project(':app').compileDebugAidl
  new Visitor(hasher, fileHasher, task).visit(task.getClass().getClassLoader())
}

class Visitor extends ClassLoaderVisitor {
    final ClassLoaderHierarchyHasher hasher
    final String prefix
    final FileHasher fileHasher
    int counter

    Visitor(ClassLoaderHierarchyHasher hasher, fileHasher, Task task) {
        this(hasher, fileHasher, "${task.path}-classloader")
    }

    Visitor(ClassLoaderHierarchyHasher hasher, FileHasher fileHasher, String prefix) {
        this.hasher = hasher
        this.prefix = prefix
        this.fileHasher = fileHasher
    }

    private String classloaderHash(ClassLoader loader) {
        def hash = hasher.getClassLoaderHash(loader)
        if (hash != null) {
            return hash.toString()
        } else {
            null
        }
    }

    @Override
    void visit(ClassLoader classLoader) {
        def hash = classloaderHash(classLoader)
        if (hash) {
            def classloaderName = classLoader.getClass().getSimpleName()
            println "${prefix}-${counter++}-${ classloaderName}-hash:  $hash"
            if (classLoader instanceof URLClassLoader && (!classloaderName.contains('ExtClassLoader'))) {
                println "${prefix}-${counter - 1}-classpath ${classLoader.getURLs().join(':')}"
                classLoader.getURLs().each { url ->
                    println "${url}: ${fileHasher.hash(new File(url.toURI()))}"
                }
            }
        }
        super.visit(classLoader)
    }
}

fab1an · September 26, 2017, 12:39pm

FileHasher can’t be resolved…

Stefan_Wolf · September 26, 2017, 1:06pm

Strange. I just tried it and it works for me. Did you copy all of the above to the top-level of your build file? In which line does the problem (“FileHasher can’t be resolved”) occur?

Stefan_Wolf · September 29, 2017, 6:40am

@fab1an Any chance on running my latest diagnostics script or reporting the error which occurred for you?

fab1an · September 29, 2017, 9:36am

Not at the moment, I’ll get back at you next week.

fab1an · October 2, 2017, 7:42am

It says that import org.gradle.internal.hash.FileHasher cannot be resolved.

I replaced the relevant line with:

println "${url}: ${HashUtil.sha1(new File(url.toURI())).asHexString()}"

and sent you the results per mail.