I am transitioning a build system to use Docker as the “glue” at the highest level, to integrate builds of components that don’t use Gradle at a local level. To achieve this, I am using custom tasks, specifying task inputs and outputs via decorated task methods, and using the Gradle build cache. This works fantastically for anything that’s producing files.
Where I just realized this breaks down is when the output of a component build is a Docker image. I want to treat that Docker image as a task output, track it as a task input, and use in the Gradle build cache just like every other file. However, there’s no file for these images; they only exists as entities within Docker.
There are some plugins out there to deal with Docker, but looking at them quickly, their focus is on building Docker images. Are there any recommendations on integrating Docker images with custom tasks to allow for up-to-date calculations and use in the Gradle build cache?
how about you use the hash (provided by docker) of the docker image as an input to the task? The docker image itself would reside in docker itself, so that wouldn’t be managed by Gradle (not shared via the build cache). The task which has the image hash as an input would then ask docker to download the corresponding image/create the corresponding container.
You could also use docker save/load or docker import/export. Given that exports all layers that is maybe not what you want and may be slower.
I think that would possibly work if the Docker image were only an input. But I have a configuration where I have a task in a project that can product a Docker image, and another task (in another project) that can consume that Docker image. Additionally, I can’t really determine what the hash of the Docker image is going to be until after the Docker image is already built. That basically means that if I’m using only the Docker image hash, I have to run the first task to calculate the input to the second task, which I don’t think works, even with something like Providers.
Therefore, my current plan (which I started, then got interrupted, but an resuming just now) is to use a file on disk that’s essentially representative of my Docker image. The contents of the file will be the image hash, as you mention. The task that produces the Docker image will also then write the hash to the file. This gives Gradle something “tangible” to track. The consuming task will have the file as an input, and will identify the Docker image by reading the hash from the file.
I believe that this will essentially enable me to treat Docker images more or less like “normal” files that can be task inputs and outputs. It’s just a bummer that I need to roll this on my own, as I was unable to find anyone else having (publicly) done this.
@rjbell4 fwiw, we use the input files to the image build process as inputs to downstream tasks that use the built image. Still requires hand wiring, but no extra hash files or similar.