Caches clean up in CI

Help/Discuss
1

The documentation says:

By default, the cleanup runs in the background when the Gradle daemon is stopped or shuts down

If we consider the pipeline then whenever Gradle daemon stops the job finishes almost the very same moment and it has a low chance of cleaning up anything, this is my assumption.

Can we enforce in any way right now the proper cleanup of it?

We have properties to configure it but I’d like to enforce the clean-up run as currently my CI cache is growing though I have ways to optimize and manage it I need this possibility too

Relates to Is there a proposed way to cleanup the artifact cache?

2

Maybe have a look at the daemon logs in <GRADLE_USER_HOME>/daemon/*/*.log and search for cleanup and cleaned up to see whether cleanup is done.

The question is, what happens to your Gradle daemons after build.

If you for example use ephemeral build agents, but with a Gradle User home that is mapped into them, then you could indeed miss the cleanup, if the ephemeral agent is force-stopped without gracefully shutting down the Gradle daemon.

If you have the ephemeral agents with the Gradle User home in them, it is thrown away together with the agent image anyway and no cleanup is necessary.

If you have persistent build agents and not run with --no-daemon the daemon continues running to execute the next applicable build until it expires. When it expires, it is gracefully shut down and does the cleanup then.

3

I’m in Gitlab pipeline so the runners live after a particular build but do not guarantee how long and nothing persisted on shutdown because we persist as Gradle cache - zip and push to a remote repository. So the best would be for me: to run clean up during the Gradle daemon run on a scheduled basis. So on cache, I could persist that.

I see that with .gradle/init.d/cache-settings.gradle:

beforeSettings { settings ->
    settings.caches {
        cleanup = Cleanup.ALWAYS
    }
}

I can achieve something, so currently, my idea is to run a scheduled pipeline job where I set this and run an arbitrary gradle task that will do the clean up too. if you have any tips on how to make it nice I’d appreciate it, like a gradle task or anything to operate with the cleanup mechanism.

4

Ah, ok, I was not aware of this setting.
If it helps you, great.
I’m not aware of any way to explicitly trigger the cache cleanup.

5

Other than stopping the daemon of course, or maybe using --no-daemon which will quit the daemon after the build.