Having the ability to see what triggered dependency resolution

not-a-bug

(Jörn Huxhorn) #1

It would be nice to have the ability to pinpoint which part of the build file is causing dependency resolution.

To achieve this, I’d suggest to show a stack trace at the start of resolution if --stacktrace is enabled.

The stack trace would then automatically contain the exact line of the build file that initiated resolution.

Alternative: only show said stack trace when resolution if happening during configuration phase.

Or is there some other way of obtaining this info that I’m unaware of?


Gradle 3.4 RC1 is now available for testing
(Jörn Huxhorn) #2

Is my assumption correct that dependency resolution during configuration time is always wrong?

If that is the case, shouldn’t it cause a deprecation warning right now and an error in a future version of Gradle?

The code below prints the following in case of our huge 522 modules build:

Total time spent resolving dependencies during configuration:
22597ms

(this is nearly 50% of the total configuration time)

The culprit triggering the dependency resolutions in our case is this line of code in the spring-boot-gradle-plugin. I suspect that this plugin is responsible for even more configuration slowdown beside just the time spent on dependency resolution that is currently measured.

Without any complaining from Gradle it appears as if the configuration time of Gradle is slow whereas the actual problem is located in a third-party plugin. This isn’t ideal from a PR point of view.

ConfigurationDetectionDependencyResolutionListener

class ConfigurationDetectionDependencyResolutionListener
        implements DependencyResolutionListener, BuildListener {
    private final Logger logger
    private boolean enabled = true
    private final Map<String, Long> timestampMap = [:]
    private final Map<String, Long> resultMap = [:]
    private final java.util.concurrent.locks.Lock lock =
        new java.util.concurrent.locks.ReentrantLock()

    ConfigurationDetectionDependencyResolutionListener(Gradle gradle) {
        logger = gradle.rootProject.logger
    }
    
    // DependencyResolutionListener
    void beforeResolve(ResolvableDependencies dependencies) {
        def path = dependencies.path
        lock.lock()
        try {
            timestampMap[path] = System.currentTimeMillis()
        } finally {
            lock.unlock()
        }
    }

    void afterResolve(ResolvableDependencies dependencies) {
        def path = dependencies.path
        long millis
        lock.lock()
        try {
            if(!enabled) {
                return
            }
    
            millis = System.currentTimeMillis() - timestampMap[path]
            resultMap[path] = millis
        } finally {
            lock.unlock()
        }

        logger.warn('PERFORMANCE WARNING: Resolving {} during configuration took {}ms!',path, millis, new Throwable())
    }

    // BuildListener
    void buildStarted(Gradle gradle) {
    }

    void settingsEvaluated(Settings settings) {
    }

    void projectsLoaded(Gradle gradle) {
    }

    void projectsEvaluated(Gradle gradle) {
        long accumulated = 0
        lock.lock()
        try {
            enabled = false
            if(resultMap.isEmpty()) {
                return
            }
    
            resultMap.each {
                accumulated += it.value
            }
            timestampMap.clear()
            resultMap.clear()
        } finally {
            lock.unlock()
        }

        logger.warn('### Total time spent resolving dependencies during configuration:\n### {}ms', accumulated)
    }

    void buildFinished(BuildResult result) {
    }
}

gradle.addListener(new ConfigurationDetectionDependencyResolutionListener(gradle))

(Stefan Oehme) #3

Build scans expose this information and add a warning in the suggestions tab.

Indeed resolution at configuration time is almost universally a design error. Even then 22s is excessive. Seeing some profiling data for this would be helpful.


(Jörn Huxhorn) #4

Something is definitely very funky with our company build. Unfortunately, we can’t do a build scan on our internal stuff.

One of the problems is that neither of my personal projects has similar problems while the company build has a less than ideal structure caused by too many people doing their own “clever” stuff… so reproducing the problem isn’t all that easy.

I’ll try to get to the bottom of the issue next week. What would such a profiling session look like?


(Stefan Oehme) #5

Just drop me a mail at stefan@gradle.com and I can send you some instructions. I don’t want to promote the tools used too much yet since they are constantly in flux.