Clean in build run through Tooling API removes also classes which are added in classpath

Old Forum
1

When running internal build using tooling API (for testing), clean removes classes not only from the test project but also all classes which are added on classpath.

We are working on a set of gradle plugins - (recently open-sourced, previously developed in-house) Ameba build framework (Apphance MobilE Build automation) - the documentation is not yet written almost at all (will be updated this week) but you can take a look at some initial pages here http://ameba.apphance.com. The source code is available at github: https://github.com/apphance/Apphance-MobilE-Build-Automation.

We run a lot of automated tests on various levels (currently around 100 of tests) but we have a small problem with tooling API-run tests. We have some test projects inside the code (https://github.com/apphance/Apphance-MobilE-Build-Automation/blob/master/testProjects/android/build.gradle) which - in order to speed up test execution use classes generated in the main project (by adding …/…/build/classes/main/ to classpath. This way we can directly run tests there without compiling the project - we have eclipse project that has output set to the same directories as gradle output, so it is really nice to be able to run tests immediately. We also run it on jenkins same way and we check code coverage with emma so we use the same technique to point to emma-instrumented classes.

All is good if we are running “gradle” binary in the test project directory. Tests are running (slow but they are running with 100% success). The trouble is when we try to run it using tooling API (which is obviously much better approach). When, in the test project we run the test project’s ‘clean’ target (standard clean target delivered by Java" we get the MAIN project classes deleted from …/…/build/main/classes directory (!), this means that next target does not find the classes for our plugins and the rest of the tests fail miserably with missing plugins.

I tried various ways to link to the classes from …/…/build/main/classes and so far only addding dependency classpath worked. Is this a bug in the clean plugin or should I link to these classes somewhat differently? I think there might be two reasons why clean removes the …/…/build/ classes -> a) either it removes classes from classpath, or b) it removes classes from build directory of the main project even if tooling api runs it in the context of another project…

The code snippets: https://github.com/apphance/Apphance-MobilE-Build-Automation/blob/master/testProjects/android/build.gradle

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath files("../../build/tmp/emma/instr/","../../build/tmp/emma/instr/main/",
             "../../build/classes/main", "../../build/resources/main", '../../bin')
        classpath 'emma:emma:2.0.5312'
    }
}
    apply plugin: 'ameba-project-configuration'
apply plugin: 'ameba-mercurial'
apply plugin: 'ameba-android-build'
apply plugin: 'ameba-project-release'
apply plugin: 'ameba-android-release'
  apply plugin: 'ameba-android-analysis'
apply plugin: 'ameba-android-apphance'
apply plugin: 'ameba-android-test'
apply plugin: 'ameba-android-jarlibrary'

Running the tests (note that if USE_PROCESS_EXECUTION is true - everything works fine, if false - first time clean is executed, any next task fails miserably with missing classes.

https://github.com/apphance/Apphance-MobilE-Build-Automation/blob/master/src/test/groovy/com/apphance/ameba/runBuilds/android/ExecuteAndroidBuildsTest.groovy

protected void runGradle(String ... tasks) {
        if (USE_PROCESS_EXECUTION) {
            def cmd = ['gradle']
            tasks.each { cmd << it }
            ProcessBuilder processBuilder = new ProcessBuilder()
            processBuilder.command(cmd).directory(testProject).redirectErrorStream(true)
            Process process = processBuilder.start()
            Thread outputThread = ProcessGroovyMethods.consumeProcessOutputStream(process, System.out)
            process.waitFor()
        } else {
            ProjectConnection connection = GradleConnector.newConnector().forProjectDirectory(testProject).connect();
            try {
                connection.newBuild().forTasks(tasks).run();
            } finally {
                connection.close();
            }
        }
    }
2

Hey,

Do you think the problem is due to the fact that the tooling api eagerly searches upper folders for the root project (settings.gradle, etc.) and runs in the context of entire project vs running in the context of a test project? To validate this theory you can temporarily move the test project somewhere else, e.g outside of your project and use tooling API to run tests.

If that’s the case here’re your options: -use searchUpwards(false) method on the DefaultGradleConnector. It’s private api, but at least you’ll solve an immediate problem -don’t use tooling api for tests until we solve that problem. Try GradleLauncher for the time being.

3

Hey Szczepan. Thanks for the hints.

It turned out this was my own bug. Maybe there is something that could caught and communicated by gradle (to warn such lame coders as I was with this). I tried your solution and it did not help but along the way I looked into the code and it turned out that I was doing some custom clean and it deleted “build” directory but the File object used was created like that:

buildDir = new File('build')

where it should be:

buildDir = new File(project.rootDir,'build')

or better

project.file('build')

Obviously - when you run it via tooling api, the current working dir remains as it was initially rather than changes to the project dir. Though maybe it could be changed (but afaik in Java it’s not possible to change working directory reliably)…

4

Cool :slight_smile:

BTW. new File() is dangerous, one might run a build from a different folder, i.e. by pointing the gradle file or project dir via a command line option. We always recommend project.file(…) instead which is much better.

Cheers!

5

Tia :slight_smile: właśnie tak jak mówisz.