Getting heap space error after upgrading to 5.5.1 gradle version from 3.5 version

14:06:59 Couldn’t flush user prefs: java.util.prefs.BackingStoreException: Couldn’t get file lock. 14:09:43 14:09:43 >

**Task :abc_core:compileTestGroovy

** 14:09:43 14:09:44 14:09:44 The system is out of resources. 14:09:44 Consult the following stack trace for details. 14:09:44 java.lang.OutOfMemoryError: Java heap space 14:09:45 at com.sun.tools.javac.code.Scope$ImportScope.makeEntry(Scope.java:530) 14:09:45 at com.sun.tools.javac.code.Scope.enter(Scope.java:220) 14:09:46 at com.sun.tools.javac.comp.MemberEnter$2.importFrom(MemberEnter.java:229) 14:09:46 at com.sun.tools.javac.comp.MemberEnter$2.run(MemberEnter.java:234) 14:09:47 at com.sun.tools.javac.comp.Annotate.flush(Annotate.java:143) 14:09:47 at com.sun.tools.javac.comp.Annotate.enterDone(Annotate.java:129) 14:09:48 at com.sun.tools.javac.comp.Enter.complete(Enter.java:512) 14:09:48 at com.sun.tools.javac.comp.Enter.main(Enter.java:471) 14:09:48 at com.sun.tools.javac.main.JavaCompiler.enterTrees(JavaCompiler.java:982) 14:09:48 at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:857) 14:09:48 at com.sun.tools.javac.main.Main.compile(Main.java:523) 14:09:49 at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129) 14:09:49 at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138) 14:09:49 at org.gradle.api.internal.tasks.compile.AnnotationProcessingCompileTask.call(AnnotationProcessingCompileTask.java:93) 14:09:49 at org.gradle.api.internal.tasks.compile.ResourceCleaningCompilationTask.call(ResourceCleaningCompilationTask.java:57) 14:09:49 at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:54) 14:09:49 at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:39) 14:09:49 at org.gradle.api.internal.tasks.compile.ApiGroovyCompiler$2$1.compile(ApiGroovyCompiler.java:258) 14:09:49 at org.codehaus.groovy.tools.javac.JavaAwareCompilationUnit.gotoPhase(JavaAwareCompilationUnit.java:94) 14:09:50 at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:548) 14:09:50 at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:517) 14:09:50 at org.gradle.api.internal.tasks.compile.ApiGroovyCompiler.execute(ApiGroovyCompiler.java:268) 14:09:50 at org.gradle.api.internal.tasks.compile.ApiGroovyCompiler.execute(ApiGroovyCompiler.java:68) 14:09:50 at org.gradle.api.internal.tasks.compile.GroovyCompilerFactory$DaemonSideCompiler.execute(GroovyCompilerFactory.java:87) 14:09:50 at org.gradle.api.internal.tasks.compile.GroovyCompilerFactory$DaemonSideCompiler.execute(GroovyCompilerFactory.java:75) 14:09:50 at org.gradle.api.internal.tasks.compile.daemon.AbstractDaemonCompiler$CompilerWorkAction.execute(AbstractDaemonCompiler.java:113) 14:09:50 at org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:47) 14:09:51 at org.gradle.workers.internal.AbstractClassLoaderWorker$1.create(AbstractClassLoaderWorker.java:46) 14:09:51 at org.gradle.workers.internal.AbstractClassLoaderWorker$1.create(AbstractClassLoaderWorker.java:36) 14:09:51 at org.gradle.internal.classloader.ClassLoaderUtils.executeInClassloader(ClassLoaderUtils.java:98) 14:09:51 at org.gradle.workers.internal.AbstractClassLoaderWorker.executeInClassLoader(AbstractClassLoaderWorker.java:36) 14:09:51 at org.gradle.workers.internal.IsolatedClassloaderWorker.execute(IsolatedClassloaderWorker.java:54) 14:09:51 startup failed: 14:09:51 Compilation failed; see the compiler error output for details.

I have already tried increasing the memory but it didn’t help.

Where did you increase the memory?
With Gradle 5.0 the daemon and all workers got restricted to 512 MiB.
As the compiler runs out of memory, I guess you just increased the daemon limit using org.gradle.jvmargs.
Instead you need to set the relevant setting on the JavaCompile task that needs more ram.

Thanks for the response.
Please can you let me know how to set the JavaCompile task for more RAM. I tried couple of things but nothing worked.
I tried-

tasks.withType(JavaCompile) {
    options.fork = true
    options.forkOptions.memoryMaximumSize = '2g'
}

tasks.withType(JavaExec) {
    jvmArgs = ['-Xms1g', '-Xmx4g']
}
compileJava {
    options.fork = true
    options.forkOptions.memoryMaximumSize = '2g' // Set the desired memory size here
}

The first and last should work fine, the first is just doing it for all compile tasks, the last for a specific one.
If you run with --debug you should see something like

[DEBUG] [org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler] Compiler arguments: -J-Xmx2g …

@Vampire Thanks for the response.

In my case JavaCompile changes didn’t work.

Changes in GroovyCompile worked.

Below code worked.

tasks.withType(GroovyCompile) {
    // Options when compiling Java using the Groovy plugin.
    // (Groovy itself defaults to UTF-8 for Groovy code)
    options.encoding = 'UTF-8'
    groovyOptions.forkOptions.memoryMaximumSize = "2g"
}

Ah, yeah, of course.
Sorry, didn’t look properly that it is a groovy compile task.