Well, we are getting more complex that I originally intended my query to me, but it seems like it might be required, and you seem to be willing to think about my situation in more depth, which I appreciate. I may have to sanitize some of this because it’s “work stuff”, and I don’t want to have to run before lawyers in order to throw up some real code, but I think we can make due; it’ll just get verbose without code examples. So here it goes…
Publication is indeed part of the plan. I kind of was waiting to try to add that on at the end (did I mention it’s a lot at once), but maybe it’s integral to the functionality. I just haven’t quite figured that part out yet, though mostly it’s because I haven’t fully invested in that part yet. Most developer builds would just publish arbitrary artifacts to a local directory, while CI builds would publish to an Artifactory repository, where dependencies could be found for later CI and developer builds, potentially by the Gradle build cache.
The overall situation here is that we have a bunch of repositories (more than a dozen, currently less than two dozen) of mostly in-house code that is builds separately, but which have dependencies between them, and which all together form a single product. We currently have a bunch of code to help link them together with a custom approach and custom scripting, but that code is … unwieldy, imperfect, and in need of replacement (and I’m being kind). I’m looking to implement much of that replacement via Gradle.
The overall workflow is that a developer would checkout one repo, or possibly more than one repo, but not necessarily every repo. We’d implement some logic to determine what revision of the all the repos we’d need, both those present, and those not present (all repos are in Git, BTW). The developer would then build the repo they checked out. For the repositories they haven’t checked out, I am hoping to use the Gradle build cache to pull down matching builds from Artifactory as dependencies. (If that doesn’t work, I can implement custom logic, but I’m hoping that the Gradle build cache can help fill this role) I have a method of calculating revisions of those dependencies without necessarily having source code present that I am optimistic I’ll be able use to with Gradle.
There might be Java parts to this code base, but it’s far from prevalent, much less universal. To my knowledge, Gradle isn’t currently in use at all. Right now, the goal is to use Gradle for the overall “glue” to join together the repositories. The build task would essentially be a “hand-off” to whatever script or build system actually builds each particular repository. I currently have that as something that’s configurable, though I’m hoping that’ll be unnecessary, and it’ll be a wrapper script of the same name and usage in every repository. We might eventually see about moving some repositories to actual Gradle builds, but that would be a more longer term item. Every repository would be buildable by the same Gradle task, which would be the default task. That is, just running
gradle gets you a build (or maybe
gw (gdub), which I plan to look at once I get things working).
So my current approach is to have a common repository with the Gradle code for every repository that’s always pulled down (we are already using a common repository in this way). I might eventually look at using Composite Builds, but at the moment am limiting my implementation to a Multi-Project Build. Part of the reason for that is (1) Composite Builds are just more stuff that add in extra complexity, which I have enough of already, and (2) I’d like to be able to give the user very helpful messages if for some reason a matching dependency cannot be found, rather than a default error message from a composite build in Gradle. But I may look at Composite Builds in a later phase, after I get something working. (I’m just mentioning it, because so far, every time I’ve left something out, you’ve recommended it! )
So in my simple prototype (mocked up repos/projects), every repository just has a very simple
settings.gradle file that defines the root project name and then applies a common settings file from the common repo. That common settings file pulls in a common custom plugin (implemented as a Groovy script plugin), and
includes the build scripts for every other repo/project that defines our overall product. The common custom plugin automatically defines common tasks, like a common build task, with specific inputs and outputs, to allow for up-to-date checks to work, and eventually also utilization of the build cache. I’m using a custom class that I add as an extension to be able to configure the dependencies (input files) and build artifacts (output files), which the common task then references. Individual projects therefore basically just need to define their inputs and outputs, and the rest is provided by the infrastructure.
That’s a lot, but I don’t know if that’s even enough to really convey the goals and current plans. But you’ve demonstrated a willingness to review the approach and provide feedback, so I’m very open to whatever feedback you can provide, and would be willing to answer any questions that you may have about our goal.