Resolving configurations in a single project results in error

Help/Discuss
1

In a single project build (such as https://github.com/kordamp/jdeps-gradle-plugin/) where the build-enforcer plugin is set in settings.gradle (https://kordamp.org/enforcer-gradle-plugin/#_applying_the_plugin) and the EnforceByteCodeVersion (https://kordamp.org/enforcer-gradle-plugin/#_enforcebytecodeversion) rule is applied an error appears on the console when invoking the build with --warning-mode all.

The EnforceByteCodeVersion eagerly resolves configurations during a callback in settings.projectsEvaluated(), before execution. The resolved configurations are copies created with configuration.copyRecursive().

The errors reported look like this when running ./gradlew clean --warning-mode all on the jdeps-gradle-plugin project

The configuration :annotationProcessorCopy was resolved without accessing the project in a safe manner.  This may happen when a configuration is resolved from a different project. This behaviour has been deprecated and is scheduled to be removed in Gradle 7.0. See https://docs.gradle.org/6.7/userguide/viewing_debugging_dependencies.html#sub:resolving-unsafe-configuration-resolution-errors for more details.

The code that performs resolution is

Now, this is a single project build. No additional projects are present, yet the error message and the linked page mentions the problem might be because another project has accessed the configuration in an unsafe manner. I’m confused now, as again, this is just a single project.

So now I’m left wondering what’s the actual cause of the problem here and how can it be fixed.

2

I’m not 100% sure how the plugin is setup but reading the code in the enforcer rule, I feel like this would also happen in a multi-project build. The relevant part is that the configuration you are resolving is not being resolved as part of the project itself. Basically a configuration “belongs to” a project and resolving it from another context, like settings, is illegal.

It looks lioe this plugin would be rather expensive to execute: wouldn’t it make sense to report only problems from actually resolved configurations during a build, and therefore just attach an afterResolve hook to the configurations you’re interested in?

3

Yes, the error is also reported in a multi-project build, but that’s not my current concern as the setup is a single project yet the error message states that the configuration was accessed in an unsafe manner (without stating how) and the link states that this could happen when crossing project boundaries, which is not the case here.

The rule those have an option to narrow configurations to be inspected, and it also resolved only those configurations that are marked as resolvable. Resolving configuration in full lazy more could be possible but kind of defeats the purpose of reporting [potential] problems early.

Your comment on “Basically a configuration “belongs to” a project and resolving it from another context, like settings, is illegal.” is perhaps why this error appears but I’ve yet to see an explanation in the docs.

If resolving a configuration can only be done by the owning project, so be it. I bit more clarity in the docs would be good IMHO.

4

Technically speaking, you are: you are resolving a project configuration from a context outside of the project.

It depends. Basically today your process is quite inefficient: it requires collecting all configurations from all projects, which potentially configures project you don’t need in a build (for example when using configure on demand). Also it will trigger, as far as I understand, the resolution of all configurations (potentially restricted to the configuration) in all projects whatever the task is called. Dependency resolution is, despite a lot of optimizations, very slow. It’s generally not an issue when it happens at execution time because we can resolve things in parallel, or different configurations from different projects in parallel. Here, you would basically be resolving always all configurations, and sequentially.

I would consider a different approach: instead of always resolving everything everytime, even when it’s not needed (say you run help or tasks or asciidoc, you could create, say, one task per project, or one task per configuration in a project, which actually resolves this configuration. There would be multiple advantages:

  • first, it would be cacheable: there’s no point in always doing the analysis if it’s always going to be the same. Modelling as a task makes it possible to optimize and cache. The result of the analysis would be a file (report or txt) that an aggregator could consume to generate a global report
  • second, the task would only be executed if needed
  • it could be, in the future, compatible with the configuration cache, meaning that all verifications could run in parallel safely

Now, of course it requires calling a task to get the feedback. You have several options here:

  1. wire the verification task to a particular lifecycle task. For example, you could decide to add it as a dependency to classes, so that everytime the classes are built, you actually execute the check first.
  2. as a variant of 1, create your own lifecycle task, so that the verification is executed only if this task is called

I don’t think it makes sense to run the analysis everytime independently of the context. It also raises the complexity of your plugin. I think that by carefully choosing the lifecycle task you hook in, you can benefit from pre-emptive checking too. It wouldn’t particularly make sense for all contexts, though, because you’re basically resolving everything without really caring in what context the configuration is used for, which forces the user to exclude configurations.

So one last approach I wanted to mention is that you could use an artifact transform instead. The idea would be that you carefully select on what configurations this has to happen. For example, when you resolve the compileClasspath or runtimeClasspath. Because then, you can alter the configuration resolution attributes to ask for a different artifact type. This would cause a transform to be executed, which, in your case, would generate the same jar if everything is fine, or fail if the bytecode isn’t what you expect. Just a rough idea of course, but again, the benefit is that the lifecyle is handled by Gradle, cached, so that if the verification has been done once for a jar, you don’t have to redo the verification.

2 Likes
5

Yes, that’s clear now but it was not before. From my POV settings is not another project, yet it’s a different context. Thus resolving configurations from a_different_context other than the owning project would be a better wording in the docs IMHO.

Efficiency is not my goal, UX is. Enforcer rules should be activated all the time, even if that affects build speed. Caching would certainly help but that requires updating the code to take advantage of that. Something that can be looked at once Configuration Cache is no longer incubating and becomes mandatory.

Hooking a enforcer task to the check task for example could work to move configuration resolution to later in the build, however that requires explicit invocation of enforcer, or check, or another task (such as build) that will invoke check as part of its dependency chain.

If there were a task that’s always invoked (say validate which would “mimic” Maven’s validate lifecycle phase, in the sense that’s invoked early in the build) then an enforcer task could be attached to it. I don’t think there is such a thing in Gradle right now, is it?

Another option would be to always inject the enforcer task into the Task Graph and have it placed as early as possible, though to be honest I don’t know if that’s possible without internal APIs.

6

You should care about performance. Perfomance is the reason why we have many users migrate from Gradle to Maven. Performance is why Gradle users ask us to implement configuration avoidance, resolution avoidance, caching: because it matters, because it’s fully part of UX. Performance is a feature. We have lots of users with large multi-project builds. You don’t want to lose those users for your plugin because you don’t care about performance.

Caching is easy and basically free if you implement it as I suggested: use a task. Configuration cache will work. I would avoid building technical debt if I could.

Precisely: the philosophy of Gradle is to do only what is required for a specific task. You can choose to ignore that, but then you are getting away of the way the tool is supposed to work. It’s going to become slow (best case) or unreliable (worst case). What’s the point of always doing the verification? By having tasks you offer more options to your users: for example they can configure CI differently. They can avoid painfully slow dependency resolution for all projects for all configurations when running tests from the IDE. Who wants to have enforcement always checking the same thing again and again if you have to pay 1 minute execution time for every invocation? That’s what you’ll end up having with the current strategy.

Enforcing is good, don’t get me wrong, but it HAS to be in context. If you really want to execute it always before compilation for example, attach it to classes, that would be enough. Or attach it to test, maybe? Alternatively you can let the user choose and let them configure when it has to execute.

2 Likes
7

Here’s a sample project which demonstrates what I mean. Look at the buildSrc directory for the implementation of the plugin.poc-enforcer.zip (112.7 KB)

8

And for completion, if you replace the enforcer.poc.gradle code with this, you get the lenient, but non cached version:

pluginManager.withPlugin('java-base') {
    configurations.configureEach { cnf ->
         cnf.incoming.afterResolve {
            // lenient, but not cached!
            it.files.each { 
                if (it.name =~ 'groovy') {
                    throw new RuntimeException("Enforcer caught ya, ${it.name}!")
                }
            }
         }
    }
}