Add report to display dependency updates


(Ben Manes) #1

A very useful feature of Maven’s dependency management are the versions:display-dependency-updates and versions:display-plugin-updates tasks. These tasks help keep a project up to date with the latest versions the libraries that it depends on, which is a healthy activity.

I am unaware of similar mechanism built into Gradle. Previously I patched the checkLibVersions example, but it is not very robust. Instead of rewriting Gradle’s excellent dependency management, I wrote a task that uses it to determine the latest versions. This follows advice given by Adam Murdoch, which I was fortunate to stumble upon.

The code below shows a working implementation that could act as inspiration for a built-in report. The dependency lookup is slow if the resolution wasn’t previously cached. This is probably because Gradle resolves each sequentially. I also found that Gradle does not always evaluate the correct latest version, so sometimes it will report “1.1” is the latest release, warning that the declared dependency is newer at 1.2.

A trimmed sample of the report is included in addition to the code.

Update 1: Added support for buildscript dependencies, no longer resolves transitive dependencies, and displays unresolved dependencies Update 2: Added support for controlling the resolution strategy (integration, milestone, release) Update 3: Added reporting for downgrading a dependency to the latest version at the specified revision level Update 4: Now evaluates a subgraphs if added to all projects and run on a specific path (root for the entire graph, a project for the subgraph).

/**
 * Adds a task that reports which dependencies have newer versions.
 * <p>
 * The <tt>revision</tt> project property controls the resolution strategy:
 * <ul>
 *
 <li>release: selects the latest release
 *
 <li>milestone: select the latest version being either a milestone or a release (default)
 *
 <li>integration: selects the latest revision of the dependency module (such as SNAPSHOT)
 * </ul>
 */
import org.gradle.api.internal.artifacts.version.LatestVersionSemanticComparator
    task dependencyUpdates(description: "Displays the dependency updates for the project.",
    group: "Help") << {
  def current = getProjectAndBuildscriptDependencies()
  def (resolved, unresolved) = resolveLatestDepedencies(current)
  def (currentVersions, latestVersions, sameVersions, downgradeVersions, upgradeVersions) =
    getVersionMapping(current, resolved)
  displayReport(currentVersions, latestVersions, sameVersions,
    downgradeVersions, upgradeVersions, unresolved)
}
    /** Returns {@link ExternalDependency} collected from the project and buildscript. */
def getProjectAndBuildscriptDependencies() {
  allprojects.collectMany{ project ->
    (project.configurations + project.buildscript.configurations).collectMany { it.allDependencies }
  }.findAll { it instanceof ExternalDependency }
}
    /**
 * Returns {@link ResolvedDependency} and {@link UnresolvedDependency} collected after evaluating
 * the latest dependencies to determine the newest versions.
 */
def resolveLatestDepedencies(current) {
  if (current.empty) {
    return [[], []]
  }
  def unresolved = current.collect { dependency ->
    dependencies.create(group: dependency.group, name: dependency.name,
         version: "latest.${revisionLevel()}") {
      transitive = false
    }
  }
  def lenient = configurations.detachedConfiguration(unresolved as Dependency[])
    .resolvedConfiguration.lenientConfiguration
  [lenient.firstLevelModuleDependencies, lenient.unresolvedModuleDependencies]
}
    /** Organizes the dependencies into version mappings. */
def getVersionMapping(current, resolved) {
  def currentVersions = current.collectEntries { dependency ->
     [label(dependency), dependency.version]
  }
  def latestVersions = resolved.collectEntries { dependency ->
     ["${dependency.moduleGroup}:${dependency.moduleName}".toString(), dependency.moduleVersion]
  }
  def sameVersions = currentVersions.intersect(latestVersions)
    def comparator = new LatestVersionSemanticComparator()
  def upgradeVersions = latestVersions.findAll { label, version ->
    comparator.compare(version, currentVersions[label]) > 0
  }
  def downgradeVersions = latestVersions.findAll { label, version ->
    comparator.compare(version, currentVersions[label]) < 0
  }
  [currentVersions, latestVersions, sameVersions, downgradeVersions, upgradeVersions]
}
    /** Returns the dependency's group and name. */
def label(dependency) { "${dependency.group}:${dependency.name}".toString() }
    /** Returns the resolution revision level. */
def revisionLevel() { project.properties.get("revision", "milestone") }
    /* ---------------- Display Report -------------- */
    /** Prints the report to the console. */
def displayReport(currentVersions, latestVersions, sameVersions,
    downgradeVersions, upgradeVersions, unresolved) {
  displayHeader()
  displayUpToDate(sameVersions)
  displayExceedLatestFound(currentVersions, downgradeVersions)
  displayUpgrades(currentVersions, upgradeVersions)
  displayUnresolved(unresolved)
}
    def displayHeader() {
  println """
    |------------------------------------------------------------
    |${project.path} Project Dependency Updates
    |------------------------------------------------------------""".stripMargin()
}
    def displayUpToDate(sameVersions) {
  if (sameVersions.isEmpty()) {
    println "\nAll dependencies have newer versions."
  } else {
    println "\nThe following dependencies are using the newest ${revisionLevel()} version:"
    sameVersions.sort().each { println " - ${it.key}:${it.value}" }
  }
}
    def displayExceedLatestFound(currentVersions, downgradeVersions) {
  if (!downgradeVersions.isEmpty()) {
    println("\nThe following dependencies exceed the version found at the "
      + revisionLevel() + " revision level:")
    downgradeVersions.sort().each { label, version ->
      def currentVersion = currentVersions[label]
      println " - ${label} [${currentVersion} <- ${version}]"
    }
  }
}
    def displayUpgrades(currentVersions, upgradeVersions) {
  if (upgradeVersions.isEmpty()) {
    println "\nAll dependencies are using the latest ${revisionLevel()} versions."
  } else {
    println("\nThe following dependencies have newer ${revisionLevel()} versions:")
    upgradeVersions.sort().each { label, version ->
      def currentVersion = currentVersions[label]
      println " - ${label} [${currentVersion} -> ${version}]"
    }
  }
}
    def displayUnresolved(unresolved) {
  if (!unresolved.isEmpty()) {
    println("\nFailed to determine the latest version for the following dependencies:")
    unresolved.sort { a, b -> label(a.selector) <=> label(b.selector) }.each {
      println " - " + label(it.selector)
      logger.info "The exception that is the cause of unresolved state:", it.problem
    }
  }
}
------------------------------------------------------------
: Project Dependency Updates
------------------------------------------------------------
    The following dependencies are using the newest release version:
 - com.google.code.findbugs:jsr305:2.0.1
 - com.google.inject.extensions:guice-multibindings:3.0
 - com.google.inject.extensions:guice-servlet:3.0
 - com.google.inject:guice:3.0
    The following dependencies exceed the version found at the release revision level:
 - org.scalatra:scalatra [2.3.0-SNAPSHOT <- 2.2.0-RC1]
 - org.scalatra:scalatra-auth [2.3.0-SNAPSHOT <- 2.2.0-RC1]
 - org.scalatra:scalatra-specs2 [2.3.0-SNAPSHOT <- 2.2.0-RC1]
    The following dependencies have newer release versions:
 - com.amazonaws:aws-java-sdk [1.3.21.1 -> 1.3.26]
 - com.beust:jcommander [1.27 -> 1.30]

(Jörn Huxhorn) #2

That task is great!


(Kevin Gaasch) #3

This task is perfect. Exactly what I was looking for!


(Ben Manes) #4

glad to hear it!

Is this a feature that the Gradle team would be willing to add to the default distributed? I think it makes sense given the dependency and dependencyInsight reports. Obviously the code above would be just proof of concept compared to native integration.

If not, I can try to package this up into a plugin so that its nicer to adopt. I haven’t written a plugin, before, though!

P.S. I’m not sure why empty lines are discarded within code tags, e.g. between methods.


(Ben Manes) #5

This is now available as a gradle plugin and should appear on Maven Central shortly (2hr sync).


(David Resnick) #6

Great plugin, Ben! Very helpful.

I have encountered a problem though where the plugin is unable to report those dependencies it failed to determine the latest version. I have a feeling that it fails for dependencies that are stored in Subversion.

The error message with stack trace is below, let me know if you need further information.

— David

Failed to determine the latest version for the following dependencies:

:dependencyUpdates FAILED

FAILURE: Build failed with an exception.

  • What went wrong:

Execution failed for task ‘:dependencyUpdates’.

No such property: group for class: org.gradle.api.internal.artifacts.ivyservice.DefaultUnresolvedDependency

  • Try:

Run with --debug option to get more log output.

  • Exception is:

org.gradle.api.tasks.TaskExecutionException: Execution failed for task ‘:dependencyUpdates’.

at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:68)

at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)

at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:34)

at org.gradle.api.internal.changedetection.CacheLockHandlingTaskExecuter$1.run(CacheLockHandlingTaskExecuter.java:34)

at org.gradle.internal.Factories$1.create(Factories.java:22)

at org.gradle.cache.internal.DefaultCacheAccess.longRunningOperation(DefaultCacheAccess.java:179)

at org.gradle.cache.internal.DefaultCacheAccess.longRunningOperation(DefaultCacheAccess.java:232)

at org.gradle.cache.internal.DefaultPersistentDirectoryStore.longRunningOperation(DefaultPersistentDirectoryStore.java:138)

at org.gradle.api.internal.changedetection.DefaultTaskArtifactStateCacheAccess.longRunningOperation(DefaultTaskArtifactStateCacheAccess.java:83)

at org.gradle.api.internal.changedetection.CacheLockHandlingTaskExecuter.execute(CacheLockHandlingTaskExecuter.java:32)

at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:55)

at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:57)

at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:41)

at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:51)

at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:52)

at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:42)

at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:247)

at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.executeTask(DefaultTaskPlanExecutor.java:52)

at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.processTask(DefaultTaskPlanExecutor.java:38)

at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:30)

at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:83)

at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)

at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)

at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)

at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:67)

at org.gradle.api.internal.changedetection.TaskCacheLockHandlingBuildExecuter$1.run(TaskCacheLockHandlingBuildExecuter.java:31)

at org.gradle.internal.Factories$1.create(Factories.java:22)

at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:124)

at org.gradle.cache.internal.DefaultCacheAccess.useCache(DefaultCacheAccess.java:112)

at org.gradle.cache.internal.DefaultPersistentDirectoryStore.useCache(DefaultPersistentDirectoryStore.java:130)

at org.gradle.api.internal.changedetection.DefaultTaskArtifactStateCacheAccess.useCache(DefaultTaskArtifactStateCacheAccess.java:79)

at org.gradle.api.internal.changedetection.TaskCacheLockHandlingBuildExecuter.execute(TaskCacheLockHandlingBuildExecuter.java:29)

at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)

at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)

at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:67)

at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)

at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:61)

at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:54)

at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:158)

at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:113)

at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:81)

at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:38)

at org.gradle.launcher.exec.InProcessGradleLauncherActionExecuter.execute(InProcessGradleLauncherActionExecuter.java:39)

at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:45)

at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:42)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:24)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.StartStopIfBuildAndStop.execute(StartStopIfBuildAndStop.java:33)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.ReturnResult.execute(ReturnResult.java:34)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:70)

at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:68)

at org.gradle.util.Swapper.swap(Swapper.java:38)

at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:68)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)

at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:59)

at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:45)

at org.gradle.launcher.daemon.server.DaemonStateCoordinator.runCommand(DaemonStateCoordinator.java:186)

at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy.doBuild(StartBuildOrRespondWithBusy.java:49)

at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.HandleStop.execute(HandleStop.java:36)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.CatchAndForwardDaemonFailure.execute(CatchAndForwardDaemonFailure.java:32)

at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:126)

at org.gradle.launcher.daemon.server.exec.DefaultDaemonCommandExecuter.executeCommand(DefaultDaemonCommandExecuter.java:48)

at org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler$ConnectionWorker.handleCommand(DefaultIncomingConnectionHandler.java:155)

at org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler$ConnectionWorker.receiveAndHandleCommand(DefaultIncomingConnectionHandler.java:128)

at org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler$ConnectionWorker.run(DefaultIncomingConnectionHandler.java:116)

at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:66)

Caused by: groovy.lang.MissingPropertyException: No such property: group for class: org.gradle.api.internal.artifacts.ivyservice.DefaultUnresolvedDependency

at com.github.benmanes.gradle.versions.DependencyUpdates.compareKeys(DependencyUpdates.groovy:177)

at com.github.benmanes.gradle.versions.DependencyUpdates$_displayUnresolved_closure14.doCall(DependencyUpdates.groovy:167)

at com.github.benmanes.gradle.versions.DependencyUpdates.displayUnresolved(DependencyUpdates.groovy:166)

at com.github.benmanes.gradle.versions.DependencyUpdates$displayUnresolved.callCurrent(Unknown Source)

at com.github.benmanes.gradle.versions.DependencyUpdates.displayReport(DependencyUpdates.groovy:115)

at com.github.benmanes.gradle.versions.DependencyUpdates$displayReport.callCurrent(Unknown Source)

at com.github.benmanes.gradle.versions.DependencyUpdates.dependencyUpdates(DependencyUpdates.groovy:49)

at org.gradle.api.internal.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:216)

at org.gradle.api.internal.BeanDynamicObject.invokeMethod(BeanDynamicObject.java:122)

at org.gradle.api.internal.CompositeDynamicObject.invokeMethod(CompositeDynamicObject.java:147)

at com.github.benmanes.gradle.versions.DependencyUpdates_Decorated.invokeMethod(Unknown Source)

at org.gradle.util.ReflectionUtil.invoke(ReflectionUtil.groovy:23)

at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$4.execute(AnnotationProcessingTaskFactory.java:161)

at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$4.execute(AnnotationProcessingTaskFactory.java:156)

at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:472)

at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:461)

at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:60)

… 78 more

BUILD FAILED


(Ben Manes) #7

Thanks David. Let’s track this on github.


(David Resnick) #8

It looks like this is already an open issue there: https://github.com/ben-manes/gradle-versions-plugin/issues/1


(Ben Manes) #9

You guys are fast! I’ll fix the groovy mistake later today (7am) and we can look into the actual unresolved problem. I’ll try to learn how to write plugin tests over the upcoming weekend.