(Is this the correct place for this post?)
In short, does it seem like a good plan for me to add an ‘isUpToDate’ method onto Task.java and onto TaskExecuter.java (so I can write custom Groovy code to compute the required build plan and add additional temporary dependencies accordingly)?
My use case is essentially the same as http://stackoverflow.com/questions/27993814/only-run-task-if-another-isnt-up-to-date-in-gradle but with many more interconnected tasks (and potentially users might want to rerun the dependent task).
We’re working on converting our existing build to Gradle, which is currently a custom script (which manages task dependencies, incrementalism and parallelism) that calls Maven.
We have a web application server that holds connections to the database server. Most of the cpu time required to deploy these applications is waiting for services to be created and such - independent of obtaining or using database connections, but the database schema must be done before the apps can finish deploying successfully.
Currently we model as different steps each of: stopping the app server, dropping a database, starting the app server, creating a database schema, deploying an app, populating the rows in a database, and checking if an app deployed successfully. Deploying one app and populating one database each take roughly one minute (though they can be parallelized, and also often skipped due to incrementalism).
Currently we have a bunch of dependencies that look like “If the database populate both is out-of-date and will run during this build, then the database schema must be run (before it) during this build too”, and similar for for schema, create-db, drop-db and stop-app-server.
We’d like a setup in Gradle that can simultaneously satisfy these scenarios:
- If the developer invokes two builds in a row, the second build runs no tasks.
- If the developer modifies the source from which mock databases are generated, then the application server is stopped, the relevant databases are dropped, the application server is restarted, the relevant databases are repopulated, and the applications are reconfirmed to be running.
It seems to me that Gradle doesn’t check whether a task is up-to-date until right before considering executing that task, which in our case causes problems because we want to preemptively stop the application server if we know we’re going to be repopulating the database.
Theoretically we could consolidate all of the database-populating into one step that first severs the connections to the application server, allowing the databases to be dropped and repopulated, but that’s not as robust in case the app server has cached database state or in case the app server tries to resave old data after the database comes back up. It also is likely to result in longer builds because many of our apps do check the schema of their databases to confirm that they can start but don’t load any rows from database until they are actually used, which means that the build can actually start checking the state of the applications after the database schema exists and before the rows have finished populating.
(The fact that our existing script displays a build plan before actually running any of the steps is also nice because it allows the developer to partially validate that they changes they made were in the right places and will show up in the build (before waiting for the build to complete). That part is similar to Gradle’s --dry-run flag except that it also distinguishes between which tasks would have in fact been run and which tasks would have been marked as UP-TO-DATE even in a non-dry run.)
I suspect that it would be a really big undertaking to modify Gradle to be able to compute this build plan beforehand (especially because in the general case, if user tasks were to create side-effects that were mistakenly not added into task.outputs, then there could be inconsistency between the expected build plan and what would actually happen with the existing Gradle where up-to-datedness is checked right before the task executes). If this were to be implemented fully into Gradle anyway, I imagine it would be an opt-in functionality (also because it probably slows down the build startup time as lots of input files are being checked) that can either be invoked by something like gradle.enablePrecomputingIncrementalBuildGraph() and then used by something like task.ifIncludedThenTrigger(otherTask).
For now what I’d like to do is to add an ‘isUpToDate’ method onto Task.java (and then implement the build planning as custom Groovy). Does that seem like a reasonable solution to this problem?