Gradle has a problem expanding large tar files?

Old Forum Archive
1

I have a 42MB tar.gz file. When I try to expand it with Gradle’s tarTree inside a Copy task:

task extractArchives(dependsOn: downloadArchives, type: Copy) {

from tarTree(‘test.tar.gz’)

into ‘dir’ }

I get this error: * What went wrong: Unable to expand TAR ‘/home/me/test.tar.gz’

The tar might be corrupted or it is compressed in an unexpected way.

By default the tar tree tries to guess the compression based on the file extension.

If you need to specify the compression explicitly please refer to the DSL reference.

  • Try: Run with --info or --debug option to get more log output.

If I create a single file with a few characters in it, TAR/GZIP it into a new test.tar.gz so it’s the only file in test.tar.gz and run my build, it works as expected. Is there some limit to the size of compressed files Gradle can handle?

2

I’m not aware of a limit, and if there is one, it’s somewhere in the GB range. There appears to be some other problem with your archive.

3

I am seeing the same issue with Gradle 2.2.1.

println tarTree(configurations.jdk.singleFile).files

Results in:

FAILURE: Build failed with an exception.

  • Where:

Build file ‘/path/to/project/build.gradle’ line: 40

  • What went wrong:

A problem occurred evaluating root project ‘project’.

Unable to expand TAR ‘/path/to/jdk/archive/jdk-8u40-linux-x64.tar.gz’

The tar might be corrupted or it is compressed in an unexpected way.

By default the tar tree tries to guess the compression based on the file extension.

If you need to specify the compression explicitly please refer to the DSL reference.

  • Try:

Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 3.081 secs

This is an unaltered copy of the Oracle JDK tarball. I have verified that it is usable from the command line. Something Gradle-related doesn’t like it. I could see it being the size (166M) as I’ve not run into this particular error before with tarballs.

In case it comes up, I get the exact same error inside a copy task :slight_smile:

4

Could you please run with ‘–stacktrace’ and provide the stack. Hopefully it contains information on the real error.

5

Exception is:

org.gradle.api.GradleScriptException: A problem occurred evaluating root project ‘project’.

at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:54)

at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:148)

at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:39)

at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:26)

at org.gradle.configuration.project.ConfigureActionsProjectEvaluator.evaluate(ConfigureActionsProjectEvaluator.java:34)

at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:59)

at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:504)

at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:83)

at org.gradle.execution.TaskPathProjectEvaluator.configureHierarchy(TaskPathProjectEvaluator.java:42)

at org.gradle.configuration.DefaultBuildConfigurer.configure(DefaultBuildConfigurer.java:35)

at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:129)

at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)

at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)

at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)

at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)

at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)

at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)

at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)

at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)

at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)

at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)

at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)

at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)

at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)

at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)

at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)

at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)

at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)

at org.gradle.launcher.Main.doAction(Main.java:33)

at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)

at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)

at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)

at org.gradle.launcher.GradleMain.main(GradleMain.java:23)

Caused by: org.gradle.api.GradleException: Unable to expand TAR ‘/path/to/jdk/archive/jdk-8u40-linux-x64.tar.gz’

The tar might be corrupted or it is compressed in an unexpected way.

By default the tar tree tries to guess the compression based on the file extension.

If you need to specify the compression explicitly please refer to the DSL reference.

at org.gradle.api.internal.file.archive.TarFileTree.visit(TarFileTree.java:79)

at org.gradle.api.internal.file.collections.FileTreeAdapter.visit(FileTreeAdapter.java:96)

at org.gradle.api.internal.file.AbstractFileTree.getFiles(AbstractFileTree.java:37)

at build_465ftqg9bnx9fjui5v1nrftif.run(/path/to/project/build.gradle:40)

at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:52)

… 32 more

Caused by: org.gradle.api.GradleException: Could not copy tar entry /path/to/jdk/archive/jdk-8u40-linux-x64.tar.gz!jdk1.8.0_40/README.html to ‘/path/to/project/build/tmp/expandedArchives/jdk-8u40-linux-x64.tar.gz_7rgta7iqynbzorxi3d85umfvr/jdk1.8.0_40/README.html’.

at org.gradle.api.internal.file.AbstractFileTreeElement.copyTo(AbstractFileTreeElement.java:79)

at org.gradle.api.internal.file.archive.TarFileTree$DetailsImpl.getFile(TarFileTree.java:121)

at org.gradle.api.internal.file.AbstractFileTree$1.visitFile(AbstractFileTree.java:39)

at org.gradle.api.internal.file.archive.TarFileTree.visitImpl(TarFileTree.java:91)

at org.gradle.api.internal.file.archive.TarFileTree.visit(TarFileTree.java:70)

… 36 more

Caused by: java.io.FileNotFoundException: /path/to/project/build/tmp/expandedArchives/jdk-8u40-linux-x64.tar.gz_7rgta7iqynbzorxi3d85umfvr/jdk1.8.0_40/README.html (Permission denied)

at org.gradle.api.internal.file.AbstractFileTreeElement.copyFile(AbstractFileTreeElement.java:91)

at org.gradle.api.internal.file.AbstractFileTreeElement.copyTo(AbstractFileTreeElement.java:74)

… 40 more

$ ls -l build/tmp/expandedArchives/jdk-8u40-linux-x64.tar.gz_7rgta7iqynbzorxi3d85umfvr/jdk1.8.0_40/README.html

-r–r--r-- 1 myuser myuser 159 Mar 20 16:05 build/tmp/expandedArchives/jdk-8u40-linux-x64.tar.gz_7rgta7iqynbzorxi3d85umfvr/jdk1.8.0_40/README.html

I notice that the file has a mode of 444. Could it just be that under the covers Gradle/Groovy/Java is trying to write to an existing read-only file without deleting it first? Not sure why it would be trying to untar it twice. For additional context, here is the relevant block of the build script up until the failing line:

configurations {

jdk

jce

}

dependencies.jdk “com.oracle.java.jdk:jdk:${version}:linux-x64@tar.gz”

dependencies.jce “com.oracle.java.jce:unlimited-jce-policy:${version[0]}@zip”

try { configurations.jdk.resolve() }

catch (org.gradle.api.artifacts.ResolveException e) {

println “We cannot build ${project.name} with version ‘${version}’. Try something like ‘8u40’”

System.exit(1)

}

println tarTree(configurations.jdk.singleFile).files