Hello,
Slightly odd situation. I always use specific versions for dependencies and therefore have never used a gradle lockfile. However, multiple SCA packages I am evaluating (suck as semgrep.dev) utilize the lockfile to perform a scan on dependencies for CVE and related issues.
So, basically, is there a way within Gradle to generate the lockfile, but disable its functionality?
(Otherwise, I will just script it as part of the CI/CD process).
Tim
As I’m like you and not use dynamic versions, I’m not fully sure.
Imho those tools should be updated to use the Gradle tooling API to get the used versions and not parse any file.
But besides that, you have to enable locking for the configurations where you want it to be used.
I would expect that if you enable it, generate the lockfile and then disable it again, that no locking is used.
Besides, what do you loose if locking is used? By disabling it you probably only risk that the lockfile and the real dependencies deviate in the future. If you really need a lockfile for those tools, it is probably best to actually use the locking, even with the fixed version.
The reason SCA tools use lockfiles is because they require minimal coding to differentiate between the many build systems, and are much more stable. I have no disagreement with that aspect.
The “hack” I would do in CI/CD would be along those lines, “find --name gradle.lockfile --delete” and then use the --write-locks while building. I would prefer a more “elegant” solution to such a brute force hack.
Tim
and are much more stable. I have no disagreement with that aspect.
I fully disagree.
Lockfiles are not meant to be stable and supported over multiple versions.
They are supported by the version of Gradle used for that build and could change any time, and rightfully though as they are just an implementation detail of the feature, not something meant to be parsed by others.
The tooling API is mainly made for other tools to integrate with Gradle like IDEs and similar.
It is designed for stability, backwards and forwards compatibility. The Tooling API version is guaranteed to support running builds with all Gradle versions for the last five major releases. For example, the Tooling API 8.0 release is compatible with Gradle versions >= 3.0. And the Tooling API is guaranteed to be compatible with future Gradle releases for the current and the next major. This means, for example, that the 8.1 version of the Tooling API will be able to run Gradle 9.x builds and might break with Gradle 10.0. So you can easily support all Gradle versions since 2016 and at least until the major version after the next major version.
By supporting lock files you can only support projects using lockfiles and thus only projects at least using 4.8 and only if you support the evolved lock files which are not guaranteed to be stable in format.
We can agree to disagree. 
It is also not material, SCA companies have implemented against the lockfile instead of the gradle tooling API.
Tim
We can agree to disagree.
Definitely.
SCA companies have implemented against the lockfile instead of the gradle tooling API.
SCA companies the SCA company you use!
If I Google for SCA solutions, first thing I find is Snyk, which is a Gradle plugin you apply in your build and run as Gradle task.
Second thing I find is Mend, which automatically adds a task to the project and executes that task, then scans the result. I didn’t check how it runs the Gradle build, but chances are very high that it uses the tooling API.
Third thing I find is OWASP Dependency Check which also is a Gradle plugin you apply and then execute a task.
Then I stopped looking.
lol, well Synk is one of the companies I am evaluating. Just not the only one.
Veracode, Semgrep (reads the lock file), and a few others.
I am actually evaluating SAST and SCA; as an “upgrade”: from PMD, Checkstyle, Spotbugs/FindBugs solution we cobbled together.
Do you really consider this an “upgrade”?
I would have thought it more as a supplement.
PMD / Checkstyle / Spotbugs check your actual code.
SCA tools check your dependencies.
At least the ones I know and have looked at in the past.
At work we for example use all those tools (OWASP Dependency Check as SCA) and additionally SonareQube which has its own rules and also integrates the findings of all those other tools in one Code Quality UI.
There is a reason I had upgrade in quotes.
There are two parts to this “upgrade”, SAST and SCA, Our existing solutions does not cover the full range of SAST requirements in PCI. Therefore, we are looking to implement a SAST tool to replace our existing solution. SAST depending on the solution either reads the source files, or the compiled binary, of our code.
Separate from SAST, we are planning to also implement SCA, while not required this year it will be soon. Therefore, we are proactively looking to implement SCA with our SAST solution later this summer. For SCA, the tool needs a dependency list to compare against CVEs and other issues. There seem to be two solutions for getting the dependency list. Look at a lock file, or integrate directly with the build tool. Hence my original question.
SonarCube is a good example, depending on the package you purchase, it can cover SAST, DAST and SCA; plus provide additional code tools such as code coverage analysis.
Tim
Even SonarQube free should cover all those parts, shouldn’t it?
Except if you need support for some of the languages only available in the paid versions, or PR decoration features, or branch analysis and so on.
To just analyse the master branch and have the issues analyzed, the static code checkers result incorporated, the code coverage incorporated, the OWASP dependency check result incorporated, … the free version is fully sufficient imho. At least that is what we use.
Hence my original question.
And did you try my idea to your original question?
The short answer is no. The OWASP dependency check does not meet our requirements. I can provide a full list of requirements if desired. But fundamentally, I am looking for a security code management tool which has capability to provide SAST and SCA. Some of the solutions for SCA that I have found depend on parsing the lock-file.
I have one POC running now, which parses a lock file for SCA. Worst case, I script around it. I have another POC starting Friday, and two more next week, of which SonarCube starts next week.
If Gradle does not have the ability to handle what I want built in (in terms of the lock-file), then this tool gets a small negative in the assessment that we have to script around it.
Let me ask again, did you try what I suggested?
You asked how to have a lockfile without it being used.
And I made a suggestion, although just based on guessing, my guessing is often quite good especially when Gradle releated. 
Ah, in your first post? No I missed that. I will try to enable and disable.
When I read that originally, I was thinking it would be easier to add --write-locks to enable it inside the CICD pipeline (both regular and merge pipelines) with a find statement before to make sure all locks are flushed.
Tim
To everyone struggling with this, I have a partial answer.
To people recommending other tools:
Semgrep’s vulnerability reachability, interface, and APIs are better than Snyk’s from what I’ve seen. Snyk is also a lot more expensive.
Going the open source route for SAST/SCA is time consuming, with a lot of work that falls on application security to dig through CVE noise and fine tune the tools, with less mature programs, this might not be an option.
You can’t change what SCA/SAST vendors require for their tools to work, and they’re all terrible in some way imo.
The following script, when added to a build.gradle file will generate new lockfiles during every build, and if there are changes, it’ll commit them as well. Since gitops in gitlab pipelines are problematic, I’ve put the onus on the dev machine to deliver new gradle lockfiles
For multiple project builds, put this in each build file, OR create a seperate file and reference it in the build file
// Automatically handle dependency locking, lockfile generation, lockfile cleanup, and git add
dependencyLocking {
lockAllConfigurations()
}
// Check for CI environment variables in the configuration phase
val ciEnvVars = listOf("CI", "GITLAB_CI", "CI_PIPELINE_ID")
val isCI = ciEnvVars.any { System.getenv(it) != null }
// Define the lockfiles task
tasks.register("lockfiles") {
if (isCI) {
logger.lifecycle("Skipping lockfiles task in CI environment.")
this.enabled = false // Disable the task in CI environment
} else {
doFirst {
// Remove the lockfile in the current directory
try {
delete(fileTree(".").matching { include("*.lockfile") })
logger.lifecycle("Deleted lockfile in the current directory.")
} catch (e: Exception) {
logger.warn("Warning: Failed to delete lockfile due to: ${e.message}")
}
// Try to generate or update lockfiles
try {
exec {
commandLine = listOf("gradle", "dependencies", "--write-locks")
}
logger.lifecycle("Lockfiles successfully updated or created.")
} catch (e: Exception) {
logger.warn("Warning: Lockfile was not created or updated due to: ${e.message}")
}
// Try to add the lockfiles to git
try {
exec {
commandLine = listOf("git", "add", "*.lockfile")
}
logger.lifecycle("Lockfiles successfully added to git.")
} catch (e: Exception) {
logger.warn("Warning: Lockfile was not added to git due to: ${e.message}")
}
}
}
}
// Make all tasks depend on 'lockfiles', except 'lockfiles' itself to prevent circular dependency
tasks.configureEach {
if (!listOf("lockfiles", "dependencies").contains(name)) {
dependsOn(tasks.named("lockfiles"))
}
}
I know OP’s specific ask was to “not honor” the lockfile, but I feel like this is where OP might have gotten if they continued to struggle with this issue.
Please, at least do not use a random available Gradle version that happens to be installed on the user’s machine, but use the tooling api to run the build, that’s what it is for and if you are within a Gradle build it also is already readily available.
Does the gradle API generate lockfiles as needed by the semgrep tooling? Can you give an example if it does? t’s not a random version being used, it’s the version specified in the build file that’s used, that snippet is for the build file. If you want something that’ll configure an env to use the correct gradle/java version in a container, I have a script for that as well
apk update && apk add --no-cache curl unzip openjdk11
# Detect Java version from build.gradle.kts or other configuration files
echo "Detecting Java version"
JAVA_VERSION=$(grep -E 'sourceCompatibility' build.gradle.kts | sed 's/.*_//g')
if [ -z "$JAVA_VERSION" ]; then
exit 0
fi
echo "Detected Java version: $JAVA_VERSION"
# Install Java
apk add openjdk${JAVA_VERSION}
# Detect Gradle version from gradle-wrapper.properties
echo "Detecting Gradle version"
GRADLE_VERSION=$(grep 'distributionUrl' gradle/wrapper/gradle-wrapper.properties | grep -oE 'gradle-([0-9.]+)-' | grep -oE '[0-9.]+')
if [ -z "$GRADLE_VERSION" ]; then
GRADLE_VERSION="7.0" # Default to Gradle 7.0 if not specified
fi
echo "Detected Gradle version: $GRADLE_VERSION"
# Install Gradle
GRADLE_ZIP="gradle-${GRADLE_VERSION}-bin.zip"
wget https://services.gradle.org/distributions/$GRADLE_ZIP
unzip $GRADLE_ZIP -d /opt
export PATH=/opt/gradle-${GRADLE_VERSION}/bin:$PATH
It’s probably not bullet proof, as it’s my specific environment, but it might help folks
You said that this task is expected to be run at the developers.
I did assume you do not do something like the snippet you showed last on each build on the developer machine, which then might have some other version of Gradle in the PATH or none at all which is the typical case.
But anyway,
Does the gradle API generate lockfiles as needed by the semgrep tooling?
I don’t know semgrep and I’m not interested in it, especially if it does not provide proper Gradle support.
But sure it can. As I said, the tooling API is the way to run or investigate a Gradle build from JVM, it is for example what also IDEs use to run Gradle, and probably is also what the Gradle CLI uses.
Ahh I see, sorry for the lack of clarity there. The original snippet is just for side loading the lockfile process into the existing build process, so it’s just for devs who are already building on their local, so there shouldn’t be any path issues. For other folks, if they’re going for the same ends, it’ll help them in their SSDLC pipelines, as the use is “invisible gradle lockfile building process”
While there are better ways to aggregate the build time dependencies that adhere more towards best practices, my only aim is to help people that were stuck in the same boat as me, with tooling that works well, but their support for gradle specific dependencies was subpar. It’s a workaround.
Sure it is, I just said, you should at least use the proper way to start the Gradle build using the tooling API instead of using an exec which is not the appropriate tool for that.