Optimizing Build Speed

Help/Discuss
1

What’s the standard pattern for keeping builds fast and optional tasks off the standard build path?

We have spotbugs, checkstyle, deployment readiness checks and springboot shadow jars which all need to build on a verification build and definitely on a build system execution but do not need to build as a part of an IDE gradle file refresh or an IDE build. Our IDE is intellij in this case.

We do the following:
task spotbugs(
group: “Verification”,
description: “”“Marker task to enabled Spotbugs (disabled by default).”""
)

gradle.taskGraph.whenReady { taskGraph ->
    tasks.spotbugsMain.onlyIf { taskGraph.hasTask tasks.spotbugs }
    tasks.spotbugsTest.onlyIf { taskGraph.hasTask tasks.spotbugs }
    tasks.spotbugsIntegrationTest.onlyIf { taskGraph.hasTask tasks.spotbugs }
}

As gradle build without spotbugs target means we do not insert spotbugs into the chain and it is skipped.

In the maven world, we’d perhaps anchor this to the verify step and would normally not see execution of these tasks in a compile/package situation. In gradle we could link these to check but that seems to run when we build.

So, how to you handle the static analysis steps which we’d like developers to run prior to PR but don’t need to execution for every file modification in the IDE?

2

None of these analysis tasks would be running when the IDE does a file refresh, and unless you’re doing something in the IDE to force different behavior or delegation not on an IDE build either. If you don’t want to run the build task frequently, don’t invoke it or add set it to run in the IDE and it won’t just decide to run.

The build task is basically everything. If you’re just wanting something like package, you should be running assemble, not build. The check is basically like verify. If you’re running the build task, it depends on both assemble and check, so you’re asking for both.

Typically, you don’t mess with with task graph, and you handle the static analysis steps by only running build or check when you actually want to do those things.

3

Hmm, so ideally I could figure out what IDEA does via the gradle tooling api and then run that with --profile on the command line to ensure only the required steps are executed. I did notice that we had some bad practices of code executing in the configuration phase vs execution which may have resulted in some slowdowns.

At this point, I’m not quite sure what IDEA does on import and on build after modification. I’m waiting on their support for a more concrete answer to that. We did find that switching it from using gradle to using its own internal strategy to improve performance for windows users.

While, I would like my team to run static analysis at some point prior to creating a PR and want the buildserver to always run it, I don’t want these steps breaking the modify/test flow. Keeping the static analysis steps optional seems like a win for us.

4

This would be of significant impact. On import, the IDE should only be configuring the build so that it can show you available tasks, recognize the source sets, and set up the appropriate classpath for dependencies. For a standard build, no actual work should be done at this point (there are edge cases where someone might want something specific run on import).

1 Like
5

The solution was made up the following:

  • Ensure nothing executes during configuration phase. This phase can set properties and configure projects. Anything else needs to happen in a target in doFirst or doLast blocks
  • Use gradle buildCache (even if it is just local) - repository managers like Nexus and Artifactory can work with the cache too. The speed up for the cache large (4m to 40s)
  • Enable parallel in gradle.properties
  • Use the daemon - even on a single use jenkins ECS node for it improves speed - especially if running multiple gradle executions within a parallel jenkins pipeline block

Thanks to @jjustinic for the feedback and suggestions.