Running unit tests with Android plugin in parallel mode causes error while reading dependencies

Bugs
1

When running Android unit tests with the maxParallelFork flag set to > 1 and cleaning both gradle cache (~/.gradle/caches/) AND maven cache (~/.m2/repository/) the build succeeds but the tests start failing with exception:

Unable to resolve artifact: Unable to get dependency information: Unable to read the metadata file for artifact 'org.robolectric:shadows-core:jar:16': Failed to build model from file '/Users/nicoloparolini/.m2/repository/org/robolectric/shadows-core/3.0/shadows-core-3.0.pom'.
    Error: 'no more data available - expected end tags </configuration></plugin></plugins></build></project> to close start tag <configuration> from line 131 and start tag <plugin> from line 128 and start tag <plugins> from line 119 and start tag <build> from line 105 and start tag <project> from line 2, parser stopped on TEXT seen ...</generatedSourcesDirectory>\n        </con... @133:14' for project org.robolectric:shadows-core
      org.robolectric:shadows-core:jar:3.0

    from the specified remote repositories:
      sonatype (https://oss.sonatype.org/content/groups/public/),
      central (http://repo1.maven.org/maven2)

    Path to dependency: 
        1) org.apache.maven:super-pom:pom:2.0

        Caused by:
        org.apache.maven.artifact.resolver.ArtifactResolutionException: Unable to get dependency information: Unable to read the metadata file for artifact 'org.robolectric:shadows-core:jar:16': Failed to build model from file '/Users/nicoloparolini/.m2/repository/org/robolectric/shadows-core/3.0/shadows-core-3.0.pom'.
        Error: 'no more data available - expected end tags </configuration></plugin></plugins></build></project> to close start tag <configuration> from line 131 and start tag <plugin> from line 128 and start tag <plugins> from line 119 and start tag <build> from line 105 and start tag <project> from line 2, parser stopped on TEXT seen ...</generatedSourcesDirectory>\n        </con... @133:14' for project org.robolectric:shadows-core
          org.robolectric:shadows-core:jar:3.0

        from the specified remote repositories:
          sonatype (https://oss.sonatype.org/content/groups/public/),
          central (http://repo1.maven.org/maven2)

        Path to dependency: 
            1) org.apache.maven:super-pom:pom:2.0

            Caused by:
            org.apache.maven.artifact.metadata.ArtifactMetadataRetrievalException: Unable to read the metadata file for artifact 'org.robolectric:shadows-core:jar:16': Failed to build model from file '/Users/nicoloparolini/.m2/repository/org/robolectric/shadows-core/3.0/shadows-core-3.0.pom'.
            Error: 'no more data available - expected end tags </configuration></plugin></plugins></build></project> to close start tag <configuration> from line 131 and start tag <plugin> from line 128 and start tag <plugins> from line 119 and start tag <build> from line 105 and start tag <project> from line 2, parser stopped on TEXT seen ...</generatedSourcesDirectory>\n        </con... @133:14' for project org.robolectric:shadows-core

                Caused by:
                org.apache.maven.project.ProjectBuildingException: Failed to build model from file '/Users/nicoloparolini/.m2/repository/org/robolectric/shadows-core/3.0/shadows-core-3.0.pom'.
                Error: 'no more data available - expected end tags </configuration></plugin></plugins></build></project> to close start tag <configuration> from line 131 and start tag <plugin> from line 128 and start tag <plugins> from line 119 and start tag <build> from line 105 and start tag <project> from line 2, parser stopped on TEXT seen ...</generatedSourcesDirectory>\n        </con... @133:14' for project org.robolectric:shadows-core

                    Caused by:
                    java.io.EOFException: no more data available - expected end tags </configuration></plugin></plugins></build></project> to close start tag <configuration> from line 131 and start tag <plugin> from line 128 and start tag <plugins> from line 119 and start tag <build> from line 105 and start tag <project> from line 2, parser stopped on TEXT seen ...</generatedSourcesDirectory>\n        </con... @133:14

The files are however present and in good shape. Running the same test suite without parallel mode works fine.

Expected result: the tests should run with both parallel mode enabled and disabled.

Steps to reproduce:
Create a simple test project following these instructions:
http://robolectric.org/getting-started/
http://robolectric.org/writing-a-test/

Use this as your build.gradle for the app module:

apply plugin: 'com.android.application'

android {
    compileSdkVersion 21
    buildToolsVersion "21.1.2"

    defaultConfig {
        applicationId "com.robolectrictest.robotest"
        minSdkVersion 21
        targetSdkVersion 21
        versionCode 1
        versionName "1.0"
    }

    testOptions.unitTests.all {
        testLogging {
            events "failed", "skipped"
            showStackTraces = true
            exceptionFormat = "full"
        }
        maxParallelForks = 8
        forkEvery = 20
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    testCompile 'junit:junit:4.12'
    testCompile 'org.robolectric:robolectric:3.0'
    androidTestCompile 'junit:junit:4.12'
}

Then create more test classes to trigger parallel mode (you can use the following python script)

for j in range(1, 200):

    testClassStart = """package com.robolectrictest.robotest;

    import android.content.Intent;
import org.robolectric.RobolectricTestRunner;
import org.robolectric.annotation.Config;
    import junit.framework.Assert;
import org.robolectric.Robolectric;
    import org.junit.Test;
    import org.junit.runner.RunWith;

    /**
     * To work on unit tests, switch the Test Artifact in the Build Variants view.
     */
@RunWith(RobolectricTestRunner.class)
@Config(constants = BuildConfig.class)
    public class ExampleUnitTest""" + str(j) + """ {"""

    testClassEnd = "}"

    testFile = open("ExampleUnitTest" + str(j) + ".java", "w")

    testFile.write(testClassStart)

    for i in range(1, 10):
        testContent = """@Test
        public void addition_isCorrect""" + str(j) + "_" + str(i) + """() throws Exception {
        MainActivity activity = Robolectric.setupActivity(MainActivity.class);
            Assert.assertTrue(true);
        }"""

        testFile.write(testContent)

    testFile.write(testClassEnd)
    testFile.close()

Try running ./gradle test, it should fail with the aforementioned exception.

I have the test project in a zipfile, but as a new user I am unable to attach it, I can provide it if needed.

Tried with gradle 2.11.

2

So that exception is coming from RoboElectric, not Gradle? In that case, the RoboElectric issue tracker would be a better place to ask about thread safety.

3

No, the exception mentions a robolectric dependency file, but it is coming from org.apache.maven when invoked by gradle. Robolectric is just the first dependency that gets its file messed up, but I’ve seen this happening with any library which might happen to be used by the currently running test.

4

Gradle does not use Maven. So this can’t come from Gradle.

So it is your tests that are pulling the dependencies while they run, right?

5

I’ve given you the ‘basic user’ privileges, so you can attach an example.

6

Here we go, this example triggers the problem when cache is cleared.
Robotest.zip (803.4 KB)

7

What I can see is that dependencies are pulled during the build, and when tests are running it fails reading xml files which have been downloaded. Those files are related to the libraries used in the tests themselves. Additionally if I manage to do a full run at least once (by means of turning parallelization off) then I am able to run them almost everytime with parallelization on (almost because once in a while they still fail).

Might it be related to Google gradle plugin rather than gradle itself?

8

As I suspected, this is caused by Roboelectric, not Gradle. Roboelectric uses a Maven dependency resolver internally. I don’t understand enough about it to tell you why.

Please ask them whether parallel execution of RoboElectric tests is supported.

9

Thank you very much, sorry for thinking it was Gradle. I will bring this to Robolectric (I wouldn’t have thought that they used a different dependency handler).

10

FYI: This is the class I’m referring to: https://github.com/robolectric/robolectric/blob/master/robolectric/src/main/java/org/robolectric/internal/dependency/MavenDependencyResolver.java

If you find something out, it would be great if you could link it back here. So in case someone has the same problem, they’ll find an answer linked here. Thanks :slight_smile:

11

Thank you. This is the issue on Robolectric:

https://github.com/robolectric/robolectric/issues/2346