JUnit Assumptions cause failing Android test

I have a “test” which takes a variety of screenshots. It has a JUnit assumption (which evaluates false by default) to make sure it isn’t run with all the rest of the tests

assumeTrue(shouldTakeScreenshots);

It seems to work everywhere except when the all-tests gradle task is run. I’ve dug through our gradle config and don’t see anything that seems like it would be a problem. (Also, we aren’t inheriting from JUnit3’s TestCase)

I’ve spent the past day hitting my head against the keyboard trying to find the cause, and haven’t had any luck. I would greatly appreciate any help - I’m a little skeptical this is actually a Gradle bug, but the rules say to post errors here.

Running it via android studio yields the following:

Running tests
Test running started
Test ignored. Assumption Failed:
org.junit.AssumptionViolatedException: got: < false>, expected: is < true>
snip (stacktrace same as below)
Finish

running the specific test via adb:

trauts-ltm:~ trauts$ adb shell am instrument -w -e shouldTakeScreenshots false -e class com.packagename.script.takeScreenshots#takeGooglePlayScreenshots com.packagename.script.test/com.packagename.script.test.ScriptAndroidJUnitRunner

com.packagename.script.takeScreenshots:
com.packagename.script.takeScreenshots:

Time: 0.02

OK (1 test)

running the test task from command line

trauts-ltm:~ trauts$ ./gradlew script:connectedAndroidTestDebug

com.packagename.script.takeScreenshots > takeGooglePlayScreenshots[emulator-5554 - 6.0] FAILED org.junit.AssumptionViolatedException: got: < false>, expected: is < true> at org.junit.Assume.assumeThat(Assume.java:95) :script:connectedAndroidTestDebug FAILED

The full stacktrace from gradle is the same as Android Studio, except android Studio considers it an ignore, while gradle says fail:

org.junit.AssumptionViolatedException: got: , expected: is
at org.junit.Assume.assumeThat(Assume.java:95)
at org.junit.Assume.assumeTrue(Assume.java:41)
at com.packagename.script.takeScreenshots.setUp(takeScreenshots.java:83)
at java.lang.reflect.Method.invoke(Native Method)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at android.support.test.internal.runner.TestExecutor.execute(TestExecutor.java:54)
at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:228)
at com.packagename.script.test.ScriptAndroidJUnitRunner.onStart(ScriptAndroidJUnitRunner.java:168)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)