Managing custom packaging tasks

Old Forum
1

I’ve developed a plugin that deals with the rather odd packaging requirements of our software, though I’m now second-guessing my design as an update to the latest milestone of gradle cut off my access to some of the dependencies. I’m basically re-implementing ZipCopySpecVisitor in the two actions as noted below, and this results in a failure to run because it can’t find org.apache.tools.zip.ZipOutputStream at runtime, which is in the 1.7.0 ant jar (the built-in version).

Structure of the package ===================

$package_name.zip
 $package_name/
  sql/
   schema/
   dataconversion/
  files/
   zip/
    webfiles.zip/
     html
     js
     ...
     WEB-INF/
      classes/
  contents.xml

Package types ============

  • full - structure as seen above * database only - omit the files/ tree from above * differential - include only files that have changed between given revisions in source control

I’ve tried to solve this with a custom package task (that extends DefaultTask) with the following actions:

  1. zip the webfiles.zip file (this is pretty much a straight Zip task, except the output is temporary)

  2. assemble the sql scripts and contents.xml of the outer zip into the $package_name.zip file. Here the visitor notes the files its visiting and adds them to the contents.xml. This is the final output of the task.

This way, the webfiles action can be skipped to produce a DB only package.

There’s the downside here of having to create that inner zip in a temp file rather than streaming it into the final archive, but I don’t know of a good way around that. Holding it in memory could be bad as this zip accounts for most of the size of the final package, and this zip can top 200 MB.

For differential packages, I determine the files to include and run an eachFile on the specs to compare the lists and exclude those that havent changed.

A convention controls the package name, the diff information (if necessary), and db-only or full. The copy specs for the subdirectories under sql/ and webfiles.zip aren’t trivial, and I would like the ability to extend/override these. Not sure if this is really a necessary feature, though.

Since I’d like to manipulate the copy specs and need to influence the zip structure from the convention, the eager-resolving CopySpecs in the Zip task haven’t worked well for me.

I’m thinking switching from org.apache.tools.zip to java.util.zip will solve my original problem, but wanted to run this by people to see if I’m going about this custom plugin in the wrong way, as most of my knowledge has come from reading through the gradle source.

2

I think your observations and conclusions are right. Offering copy specs so that the plugin client can use them to configure the contents of the package makes perfect sense.

I suggest to open source the plugin, provide documentation with samples so that other people can start using it and give you tangible feedback. Also, observe how the plugin is used in your environment and improve it based on real uses cases :slight_smile:

Hope that helps!

3

I actually took the plugin in a different direction with the help of Alkemist from the gradle IRC, but I think there is still some value in some of the ideas from this, such as a lazy-eval CopySpec or pluggable CopySpecVisitor. I may toy around with that when I get some free time. Thanks for the response.