I had a similar issue:
Process 'Gradle Test Executor 2' finished with non-zero exit value 137
I did not try to stop services as suggested by @hirscheran, since my build runs on a container with sudo: false and I didn’t want to change that.
As @sherter suggested, the problem is that GradleRunner forks processes which end up taking too much memory, then Travis kills them.
What solved the problem for me was reducing max heap size of forked gradle processes run by GradleRunner.
This can be done like this:
GradleRunner.create()
.withJvmArguments("-Xmx128m")
...
build()
Notice that the method withJvmArguments is not defined on the GradleRunner abstract class, but is defined on the DefaultGradleRunner object which is returned from GradleRunner.create(). So if your code is Java, you’ll have to cast. If your code is Groovy you are fine without casting.