How can I build gradle 3.0 purely from the source code, without using gradle binaries downloaded from the internet (as the gradlew wrapper does)? Every time I search for this in the documentation, I wind up at: “Naturally, Gradle builds itself with Gradle”, but surely there must be some supported way to bootstrap the build, or else this really shouldn’t be considered open source software.
I do appreciate the easy-to-use process implemented by the wrapper script, and that most users are happy with that, but I would like to build from pure source so as not to contaminate the build (and potentially everything gradle builds) with the unknown contents of the bootstrap binary.
Any documentation on building gradle without gradle (even an earlier version, if I could then use that to build 3.0) would be helpful.
Prepare for an answer by which I anticipate you will not be satisfied.
When you work on any open-source project, you are building the code of that project with tools and sometimes scripts which modify things during the build. It doesn’t matter if those tools are just compilers and linkers or a build-automation tool such as make, scons, ant, maven or gradle.
Once the process of building a software project becomes sufficiently complicated, it is very important that anyone contributing to the project can assemble the desired binaries without needing to understand every step of the process. This is why many open-source projects include build scripts and encourage their community to use build automation tools (like gradle) to build their software. Why should this open-source project, be any different?
You say that you are trying to avoid contaminating the build with the unknown contents of the bootstrap binary. But, consider two separate points here:
1.The contents of the bootstrap binary are not unknown. Every release of Gradle is tagged in our Git repository, so you can always checkout that tag and see exactly what the contents of that binary are.
2. At some point, you have to trust your tools. Would you feel the same way about the javac compiler? What if, we had a bash script you could run which would only invoke the javac compiler and other tools for packaging the binary distributions directly. Wouldn’t you be uncomfortable with the possibility that you are contaminating the build with the “unknown contents” of the javac compiler?
If you are concerned about actually auditing the provenance of a gradle binary: that is, making sure that the binary we are hosting for download is actually built from the state of the repository associated with that binary’s tag in our GitHub repository, we can probably provide some hints on how to confirm that.
However, attempting to build Gradle, with all of the complexity currently encoded in the gradle build scripts for this project without a version of gradle which understands those scripts is not really a good use of anyone’s time.
Yes, exactly like GCC. In addition to building GCC using itself, GCC has always supported bootstrapping from any number of other compilers so that there is a route that can be taken to build gcc itself entirely from source (using very basic C compilers such as one implemented directly in assembly).
I’m not trying to give you a hard time, and it’s true that to be realistic we do have to trust something at some point. Whether it’s a low-level compiler binary, the kernel, or the machine hardware itself most of us do not control the system all the way down. However, I was very surprised to see a high level build tool like gradle require bootstrapping from binaries. I appreciate that dogfooding is a good way to make sure gradle is fit for purpose, but I’d just like to see some (probably more complex) way of building it using other tools.
The sort of attack that (in general) can be perpetuated by not building directly from source is exemplified by Ken Thompson’s C bootstrap compiler / login backdoor, described in his paper “Reflections on Trusting Trust”: http://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf
I do have reservations about using the proprietary javac provided by Oracle/Sun. I prefer to use OpenJDK whenever possible, but sometimes code requires the Oracle JDK and we have no choice but to use that. We install such packages into a separate namespace for untrusted/blackbox software.
If it isn’t possible to compile gradle without relying on an opaque binary provided by a third-party, I guess the alternative for us will be to install gradle and everything we build using gradle into the “untrusted” area.
As far as I know, gradle is the only major java build tool for which there is apparently no supported path to compiling it without involving a third-party binary.
ant could be built simply with javac and a shell script.
ivy can be built with ant.
maven (1.x) could be built with ant, and maven 1.x used to compile later versions of maven.
I’d be happy with a circuitous route that involves compiling one or more earlier versions of gradle in order to bootstrap future versions. It seems clear that someone did compile some version of gradle without gradle at some point - if there is any information on how that was done (perhaps someone has a shell script that can build a very early version), along with which version(s) of gradle each newer version can be built with, I’d be happy to give it a try and attempt to document some sort of “pure” bootstrapping process.
Just to add something to the discussion: I took a quick look at the early commits (from 9 years ago) on GitHub, and although I didn’t understand much, I could see that it was originally using svn. Perhaps in the svn repo there might be something more useful.
Still, in the early git commits, there’s a lot of Groovy code that I think looks to be part of both Gradle and it’s own bootstrapping, and I didn’t try to understand much how the build process worked. I didn’t see anything I could identify as an ant or maven file, but as I said I just did a quick look. Anyway, these thoughts might be incorrect, as a more thorough view is needed.
Still, if true, it might be hard to do a bootstrap from an early version. Forking the git repo just to have a non-Gradle built Gradle, even on an earlier version that’s much simpler, could still mean a lot of work.
One idea that came to mind was to have a way (maybe a plugin?) to have a more capable ‘dry run’ of the build (like what Gnu Make does). Instead of actually building Gradle, it would output a script with all commands necessary to build Gradle without actually using Gradle (not just what tasks would be executed, but also their equivalent execution as shell commands). I assume much of the script would probably be curl, javac, cp, mv, and zip commands. Note that the script would be highly unportable, but that’s ok, as long as it can be executed on a clean checkout of the repo and successfully builds everything.
Don’t know how feasible such a thing would be, nor if it would be less work than the fixed-version bootstrap idea.
This will create a target/dist/gradle-1.0-snapshot directory with a gradle distribution in it. (You may need to create an empty src/samples directory in the repo to make the distribution build.) Even then, it may complain of failing tests, but you’ve got a Gradle binary.
From here, I would attempt to find some later point in the repo where you can build gradle using that gradle binary, and then keep fast-forwarding until you find a version of Gradle which can’t be built with that binary and try building it with the one you built with the first one, and then repeat. Good luck.