Can dependencies be used to manage runtime executables? If so, how?


(aaarnett) #1

I have a set of executables (not jars) that are required at build time, and I’m looking for a good solution to make them versioned/fetchable/usable without explicitly installing them on the build machines.

I would like to store the executable in either a repository, or a common fileshare.

I had hoped I would be able to declare a configuration, add a repository pointing to either the repository or the fileshare and add a dependency referencing the location.

Then at runtime, gradle would resolve (presumably downloading/caching from the repo or share) and provide access to that executable.

For example:

repositories {
  ivy {
    artifactPattern "file://<my-file-server>/Software/repo/[organization]/[revision]/[artifact].[ext]"
  }
}
  configurations {
  phantom
}
  dependencies {
  phantom group: 'phantomjs', name: 'phantomjs', version: '1.9.2-windows', ext: 'exe'
}
  task testPhantom(type: Exec) {
  def resolvedConfiguration = configurations['phantom'].resolvedConfiguration
  def executable = resolvedConfiguration.getFiles()
  commandLine executable
}

When I use this test script, the debug output includes:

[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Visiting dependency :TestGradleDependencyFileshare:unspecified(phantom) -> phantomjs:phantomjs:1.9.2-windows(dependency: phantomjs#phantomjs;1.9.2-windows {phantom=[default]})
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Selecting new module version phantomjs:phantomjs:1.9.2-windows
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.UserResolverChain] Attempting to resolve module 'phantomjs:phantomjs:1.9.2-windows' using repositories [ivy]
[DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver] Loading \<my-file-server>\Software\repo/phantomjs/1.9.2-windows/ivy.xml
[DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver] Resource not reachable for phantomjs#phantomjs;1.9.2-windows!ivy.xml(ivy): res=MissingResource: \<my-file-server>\Software\repo/phantomjs/1.9.2-windows/ivy.xml
[DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver] Loading \<my-file-server>\Software\repo/phantomjs/1.9.2-windows/phantomjs.exe
[DEBUG] [org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver] No meta-data file found for module 'phantomjs#phantomjs;1.9.2-windows' in repository 'ivy', using default data instead.
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.UserResolverChain] Using module 'phantomjs:phantomjs:1.9.2-windows' from repository 'ivy'
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Visiting configuration phantomjs:phantomjs:1.9.2-windows(default).
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] phantomjs:phantomjs:1.9.2-windows(default) has no transitive incoming edges. ignoring outgoing edges.
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Attaching :TestGradleDependencyFileshare:unspecified(phantom) to its parents.
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.DependencyGraphBuilder] Attaching phantomjs:phantomjs:1.9.2-windows(default) to its parents.
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.oldresult.TransientConfigurationResultsBuilder] Flushing resolved configuration data in Binary store in C:\Users\me\AppData\Local\Temp\gradle7270301199399607391.bin. Wrote root :TestGradleDependencyFileshare:unspecified:phantom.
[DEBUG] [org.gradle.configuration.project.BuildScriptProcessor] Timing: Running the build script took 2.914 secs
[ERROR] [org.gradle.BuildExceptionReporter]
 [ERROR] [org.gradle.BuildExceptionReporter] FAILURE: Build failed with an exception.
[ERROR] [org.gradle.BuildExceptionReporter]
 [ERROR] [org.gradle.BuildExceptionReporter] * Where:
[ERROR] [org.gradle.BuildExceptionReporter] Build file 'C:\Users\me\workspace\TestGradleDependencyFileshare\build.gradle' line: 17
[ERROR] [org.gradle.BuildExceptionReporter]
 [ERROR] [org.gradle.BuildExceptionReporter] * What went wrong:
[ERROR] [org.gradle.BuildExceptionReporter] A problem occurred evaluating root project 'TestGradleDependencyFileshare'.
[ERROR] [org.gradle.BuildExceptionReporter] > Could not resolve all dependencies for configuration ':phantom'.
[ERROR] [org.gradle.BuildExceptionReporter]
  > java.lang.NullPointerException (no error message)

This indicates to me that gradle was able to resolve the exe, but not the ivy xml file (which does not exist, is presumably not required and defaults will be used), yet the ultimate error is still ‘Could not resolve all dependencies’…

Is there something else at play in the resolution of the executable? Or do I really need ivy descriptors for file-based repos?

Additionally, since the resolve is failing don’t know what will happen when it actually hits the exec, is this the best way to resolve and address the dependency, or is there a cleaner way?

Thanks, Aaron


(Luke Daley) #2

This should work.

What version of Gradle?


(aaarnett) #3

I’m using 1.10


(Luke Daley) #4

Can you run with -s and provide the stacktrace please.


(aaarnett) #5

Certainly…

[ERROR] [org.gradle.BuildExceptionReporter] FAILURE: Build failed with an exception.
[ERROR] [org.gradle.BuildExceptionReporter]
 [ERROR] [org.gradle.BuildExceptionReporter] * Where:
[ERROR] [org.gradle.BuildExceptionReporter] Build file 'C:\Users\me\workspace\TestGradleDependencyFileshare\build.gradle' line: 17
[ERROR] [org.gradle.BuildExceptionReporter]
 [ERROR] [org.gradle.BuildExceptionReporter] * What went wrong:
[ERROR] [org.gradle.BuildExceptionReporter] A problem occurred evaluating root project 'TestGradleDependencyFileshare'.
[ERROR] [org.gradle.BuildExceptionReporter] > Could not resolve all dependencies for configuration ':phantom'.
[ERROR] [org.gradle.BuildExceptionReporter]
  > java.lang.NullPointerException (no error message)
[ERROR] [org.gradle.BuildExceptionReporter]
 [ERROR] [org.gradle.BuildExceptionReporter] * Exception is:
[ERROR] [org.gradle.BuildExceptionReporter] org.gradle.api.GradleScriptException: A problem occurred evaluating root project 'TestGradleDependencyFileshare'.
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:54)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:156)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:38)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:25)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.configuration.project.ConfigureActionsProjectEvaluator.evaluate(ConfigureActionsProjectEvaluator.java:34)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:55)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:507)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:82)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.configuration.DefaultBuildConfigurer.configure(DefaultBuildConfigurer.java:31)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:142)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:113)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:81)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:64)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:35)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:50)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:201)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:174)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:170)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:139)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.Main.doAction(Main.java:46)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.Main.main(Main.java:37)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:50)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:32)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:58)
[ERROR] [org.gradle.BuildExceptionReporter] Caused by: org.gradle.api.artifacts.ResolveException: Could not resolve all dependencies for configuration ':phantom'.
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingArtifactDependencyResolver.wrapException(ErrorHandlingArtifactDependencyResolver.java:57)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingArtifactDependencyResolver.access$000(ErrorHandlingArtifactDependencyResolver.java:34)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingArtifactDependencyResolver$ErrorHandlingResolvedConfiguration.getFiles(ErrorHandlingArtifactDependencyResolver.java:186)
[ERROR] [org.gradle.BuildExceptionReporter]
at build_qeks69b41j7kprviqku0jh1og$_run_closure4.doCall(C:\Users\me\workspace\TestGradleDependencyFileshare\build.gradle:17)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:58)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:130)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:110)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.AbstractTask.configure(AbstractTask.java:439)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.project.AbstractProject.task(AbstractProject.java:958)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:246)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.BeanDynamicObject.invokeMethod(BeanDynamicObject.java:134)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.CompositeDynamicObject.invokeMethod(CompositeDynamicObject.java:147)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.groovy.scripts.BasicScript.methodMissing(BasicScript.java:79)
[ERROR] [org.gradle.BuildExceptionReporter]
at build_qeks69b41j7kprviqku0jh1og.run(C:\Users\me\workspace\TestGradleDependencyFileshare\build.gradle:15)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:52)
[ERROR] [org.gradle.BuildExceptionReporter]
... 33 more
[ERROR] [org.gradle.BuildExceptionReporter] Caused by: java.lang.NullPointerException
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.util.CollectionUtils.filter(CollectionUtils.java:86)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.util.CollectionUtils.filter(CollectionUtils.java:59)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.artifacts.ivyservice.SelfResolvingDependencyResolver$SelfResolvingFilesProvider.getFiles(SelfResolvingDependencyResolver.java:61)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.artifacts.ivyservice.SelfResolvingDependencyResolver$FilesAggregatingResolvedConfiguration.getFiles(SelfResolvingDependencyResolver.java:80)
[ERROR] [org.gradle.BuildExceptionReporter]
at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingArtifactDependencyResolver$ErrorHandlingResolvedConfiguration.getFiles(ErrorHandlingArtifactDependencyResolver.java:184)
[ERROR] [org.gradle.BuildExceptionReporter]
... 45 more
[ERROR] [org.gradle.BuildExceptionReporter]
 [LIFECYCLE] [org.gradle.BuildResultLogger]
 [LIFECYCLE] [org.gradle.BuildResultLogger] BUILD FAILED
[LIFECYCLE] [org.gradle.BuildResultLogger]
 [LIFECYCLE] [org.gradle.BuildResultLogger] Total time: 12.907 secs

Thanks!


(aaarnett) #6

Minor update:

I tested this with today’s 1.11 release, with the same results. I’ve also tested on the local filesystem (as opposed to fileshare) with the same results.

Thanks!


(aaarnett) #7

It seems this thread was dropped… Was the stack trace provided sufficient? Is there something further needed to continue the conversation? Feedback on this issue is appreciated…


(Luke Daley) #8

Ok, I see the problem.

You just want ‘configurations[‘phantom’].files’, instead of going through the resolved configuration.


(aaarnett) #9

I’ve finally gotten back to this and can confirm that referencing as you suggest works.

The one additional piece is that if this is being used directly/explicitly - as in an Exec task - the LinkedHashSet has to be further resolved. Thus…

configurations['phantom'].files.iterator().next()

Thanks for your help!


(Peter Niederwieser) #10

Or ‘configurations[‘phantom’].singleFile’.


(aaarnett) #11

Excellent tip… I was hoping there was a more concise usage.

Thanks!