Trying again to get a custom javadoc taglet to work

Quite a while ago, I started the process of trying to convert a largish multiproject Maven build to Gradle, partly to compare the resulting benefits, but at the time I did it because I was at a standstill with something that I was trying to implement in the Maven build. I concluded it was just too onerous to do in Maven, so I decided it was time to try converting.

Note that I’ve posted about my issues with this in the recent past, but I haven’t gotten any answers that address my problem, so this effort has simply been blocked for the last few weeks. I’m going to try again, perhaps posting more code, to see what answers I can get. My real issues may all be with a particular plugin that I’m trying to use, but I got no response to the issue I filed for that (Not emitting -tagletpath option · Issue #15 · nebula-plugins/gradle-aggregate-javadocs-plugin · GitHub).

My issue is with a custom javadoc taglet that I wrote. The thing that makes this taglet unusual is that it needs to get access to the class file corresponding to the source file it is contained within. Implementing this “properly” in the Maven build required doing something that I refuse to do (encode information about all the child artifacts in the parent pom).

In both the Maven and Gradle builds, getting this to work in a single project wasn’t that onerous. Getting it to work in the “aggregated” javadoc task is where I have trouble. As far as I can tell, “gradle-aggregate-javadocs-plugin” is what I need to use to aggregate all of the javadoc from all of the projects into a single output. This is what I can’t get to work, or I’m just not using it correctly.

I’m including here my top-level build.gradle file, removing some irrelevant lines:

buildscript {
    repositories { jcenter() }

    dependencies {
        classpath 'com.netflix.nebula:gradle-aggregate-javadocs-plugin:2.2.+'
    }
}

apply plugin: 'nebula-aggregate-javadocs'

tasks.withType(Javadoc) {
    options {
        encoding    = "UTF-8"
        addStringOption('Xdoclint:none', '-quiet')
        addStringOption("Dorg.apache.logging.log4j.level=debug")
        taglets "com.att.det.taglet.ValidationConstraintsTaglet", "com.att.det.taglet.ValidationConstraintsCombinedTaglet"
        addStringOption("top").value  = "Unified Service Layer - top"
        bottom "Unified Service Layer - bottomxx"
        windowTitle "Unified Service Layer - windowtitlexx"
        docTitle "Unified Service Layer - titlexx"
        footer "Unified Service Layer - footerxx"
        header "Unified Service Layer - headerxx"
        setMemberLevel JavadocMemberLevel.PUBLIC
        afterEvaluate {
            tagletPath ((configurations.taglet.files) as File[])
        }
    }
}

afterEvaluate {
    def allOutputs =
    subprojects.stream().
    map { !(it.name in ["jacoco-aggregate"]) ? it.sourceSets.main.output : Collections.emptySet() }.
    collect();

    println "allOutputs[${allOutputs}]"
}


allprojects  {
    apply plugin: 'maven'

    group = 'com.att.detsusl'
    version = '2.7.0-SNAPSHOT'
}

configurations {
    taglet
}

dependencies {
    taglet "com.att.detsusl.taglets:validationJavadocTaglet:0.0.1-SNAPSHOT"
}

subprojects {
    apply plugin: 'java'
    
    sourceCompatibility = 1.8
    targetCompatibility = 1.8
    
    tasks.withType(JavaCompile) {
        options.encoding = 'UTF-8'
    }

    repositories {
        mavenLocal()

        maven { url "http://mavencentral.it.att.com:8084/nexus/content/repositories/att-public-group/" }
        maven { url "http://mavencentral.it.att.com:8084/nexus/content/repositories/att-repository-snapshots" }
        maven { url "http://repo.maven.apache.org/maven2" }
    }

    configurations {
        taglet
    }
    
    dependencies {
        compile "org.apache.commons:commons-lang3:3.5"
        compile "commons-lang:commons-lang:2.6"
        ["jackson-annotations", "jackson-core", "jackson-databind"].each {
            compile "com.fasterxml.jackson.core:$it:2.8.3"
        }
        compile "com.google.code.gson:gson:2.8.0"
        compile "log4j:log4j:1.2.17"
        compile "org.apache.aries.blueprint:org.apache.aries.blueprint.annotation.api:1.0.0"
        compile "org.apache.aries.blueprint:org.apache.aries.blueprint.api:1.0.0"
        compile "com.sun.xml.bind:jaxb-core:2.2.11"
        compile "com.sun.xml.bind:jaxb-impl:2.2.11"
        compile "org.springframework:spring-context:4.2.5.RELEASE"
        
        taglet "com.att.detsusl.taglets:validationJavadocTaglet:0.0.1-SNAPSHOT"
        
        ["pax-exam", "pax-exam-container-karaf", "pax-exam-junit4"].each {
            testCompile "org.ops4j.pax.exam:$it:4.9.1"
        }
        testCompile "org.powermock:powermock-api-mockito:1.6.6"
        testCompile "org.assertj:assertj-core:3.6.2"
        testCompile "commons-jxpath:commons-jxpath:1.3"
    }

    test {
        exclude '**/*IT.*'
        exclude '**/*PaxConfigurer.*'
    }

    if (!(project.name in ["javadoc-aggregate"])) {  
        javadoc {
            options {
                encoding = 'UTF-8'
                addStringOption("top").value  = "Unified Service Layer - top"
                bottom "Unified Service Layer - bottom"
                windowTitle "Unified Service Layer - windowtitle"
                docTitle "Unified Service Layer - title"
                footer "Unified Service Layer - footer"
                header "Unified Service Layer - header"
                setMemberLevel JavadocMemberLevel.PUBLIC
                addStringOption('Xdoclint:none', '-quiet')
                addStringOption("Dorg.apache.logging.log4j.level=debug")
                
                taglets "com.att.det.taglet.ValidationConstraintsTaglet", "com.att.det.taglet.ValidationConstraintsCombinedTaglet"

                afterEvaluate {
                    tagletPath ((configurations.taglet.files + sourceSets.main.output) as File[])
                }
            }
        }
    }
}

Note again that this works fine for the “javadoc” task in each project. The taglet works fine, finding the class associated with each source file.

When I run the “aggregateJavadocs” task, I see this in the output:

javadoc: error - Error - Exception java.lang.ClassNotFoundException thrown while trying to register Taglet com.att.det.taglet.ValidationConstraintsTaglet…
javadoc: error - Error - Exception java.lang.ClassNotFoundException thrown while trying to register Taglet com.att.det.taglet.ValidationConstraintsCombinedTaglet…

What’s annoying about these error messages is that it’s not saying WHICH class is not found. I can’t tell whether it can’t find the taglet class, or the class associated with the source file. The fact that it refers to the registration process seems to imply that it’s the taglet class itself, but I’m not sure.

If I look closer at the resulting command-line options to “javadoc”, it provides a big clue that shows why it can’t find the taglet classes.

I opened up the “build/tmp/javadoc/javadoc.options” file for one of the subprojects, and it has the proper “-tagletpath” option, in addition to “-taglet” and the options to set the various string fields. When I opened up the “javadoc.options” file in the parent, from the “aggregateJavadocs” task, no “-tagletpath” option was present. That’s the only way the javadoc executable can find the taglet classes, not to mention the class files associated with the source files.

it’s entirely possible I’m simply specifying the “tagletpath” option wrong in the build.gradle file, but I can’t see that.

This issue is probably due to the usage of

afterEvaluate {
            tagletPath ((configurations.taglet.files) as File[])
        }

You shouldn’t use afterEvaluate.

When you apply gradle-aggregate-javadocs-plugin, it will register a clousre with projectsEvaluated here, just like you do with afterEvaluate. This is the interesting part: the closure gradle-aggregate-javadocs-plugin registered would be executed before your closure, so it couldn’t see your tagletPath.

I don’t know why you are using an afterEvaluate, but it may be the cause.

I’ve created an example to compare the usage with and without afterEvaluate here: https://github.com/blindpirate/multi-doc

Please let me know if this solves your question.

Ok. Thanks for replying. I saw the reply on Friday, but I haven’t had time to work on it until now.

Taking the “tagletPath” setting out of the “afterEvaluate” block in the top-level javadoc block appears to have resulted in it now emitting the “-tagletpath” command-line option. However, this may have been the easier part of the problem.

Despite the “-tagletpath” option now being on the command line, the processing still fails with exceptions like this:

java.lang.ClassNotFoundException: com.att.det.usl.fraudcheck.api.ExecuteFraudCheckRequest
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.att.det.taglet.ValidationConstraintsTaglet.toString(ValidationConstraintsTaglet.java:76)

This is happening because the “-tagletpath” is not including the class files from the compiled source. This shouldn’t be that surprising, as the setting as I have it there is not attempting to do that. I have to figure out how to make that happen.

If you look in the SECOND javadoc settings block in the build script, you’ll see the following block:

            afterEvaluate {
                tagletPath ((configurations.taglet.files + sourceSets.main.output) as File[])
            }

These are the settings for the “javadoc” task in each subproject. When “javadoc” runs in each subproject, this results in the “-tagletpath” command-line option having all the required jars, combining the classes for the taglet AND the class files from the compiled source. As you can see, this is in an “afterEvaluate” block. I don’t remember the history of this, but I believe this particular block required “afterEvaluate” to work.

So what I have to figure out is how to change the “tagletPath” setting in the top-level javadoc settings block to include the “sourceSets.main.output” for EVERY subproject in the “tagletPath” setting.

Well, I don’t think you need afterEvaluate at all. Once java plugin is applied, sourceSets.main.output is available. I’ve updated https://github.com/blindpirate/multi-doc , if you run gradle aggregateJavadocs -PsetPathNow=true again, you would see the following content in project a’s javadoc.option:

-classpath '/Users/zhb/Projects/tmp/multidoc/a/build/classes/java/main:/Users/zhb/Projects/tmp/multidoc/a/build/resources/main'
-d '/Users/zhb/Projects/tmp/multidoc/a/build/docs/javadoc'
-doctitle 'a API'
-quiet
-taglet 'foo.ToDoTaglet'
-tagletpath '/Users/zhb/Projects/tmp/multidoc/taglet.jar:/Users/zhb/Projects/tmp/multidoc/a/build/classes/java/main:/Users/zhb/Projects/tmp/multidoc/a/build/resources/main'
-windowtitle 'a API'
'/Users/zhb/Projects/tmp/multidoc/a/src/main/java/a/A.java'

I guess this is what you want.

There may be some confusion here. My top-level build script has two javadoc settings blocks. One at the “top-level”, which configures the “aggregateJavadocs” task. The other is inside the “subprojects” block, which configures the “javadoc” task in each subproject.

The subproject javadoc task works perfectly fine, which is configured by the subproject javadoc block, which has the “afterEvaluate” block. I can experiment with removing that, but that is not the part that I have an issue with.

My problem is with the top-level javadoc block, and the resulting “aggregateJavadocs” task. With your help, I’ve made the change that resulted in at least emitting a “-tagletpath” command-line option, by removing the “afterEvaluate” wrapper. However, the more difficult step is making it emit the correct contents for that option. As the subproject javadoc block appends “sourceSets.main.output” to the tagletPath, the top-level javadoc block needs to append something like (not real syntax) “allProjects.sourceSets.main.output”. I imagine this will have to be some sort of stream mechanism, to produce a list of “sourceSets.main.output” entries for each subproject.

I also wonder whether the code that processes the subproject javadoc block works differently from the code in the Nebula javadoc plugin, as the subproject javadoc works fine with the “afterEvaluate” block, but the top-level one does not.

Okay, I understand. A solution is

tasks.whenTaskAdded { task ->
    if('aggregateJavadocs' != task.name) {
        return
    }
    aggregateJavadocs {
        options {
            taglets 'foo.ToDoTaglet'
            def path = configurations.taglet.files
            subprojects.findAll { subproject -> subproject.plugins.hasPlugin(JavaPlugin) }.each {path = path + it.sourceSets.main.output}
            tagletPath (path as File [])
        }
    }
}

When root project’s aggregateJavadocs task is created, the closure will configure it with all subprojects’ classpaths. The result options file is:

-classpath '/Users/zhb/Projects/tmp/multidoc/a/build/classes/java/main:/Users/zhb/Projects/tmp/multidoc/a/build/resources/main:/Users/zhb/Projects/tmp/multidoc/b/build/classes/java/main:/Users/zhb/Projects/tmp/multidoc/b/build/resources/main'
-d '/Users/zhb/Projects/tmp/multidoc/build/docs/javadoc'
-quiet
-taglet 'foo.ToDoTaglet'
-tagletpath '/Users/zhb/Projects/tmp/multidoc/taglet.jar:/Users/zhb/Projects/tmp/multidoc/a/build/classes/java/main:/Users/zhb/Projects/tmp/multidoc/a/build/resources/main:/Users/zhb/Projects/tmp/multidoc/b/build/classes/java/main:/Users/zhb/Projects/tmp/multidoc/b/build/resources/main'
'/Users/zhb/Projects/tmp/multidoc/a/src/main/java/a/A.java'
'/Users/zhb/Projects/tmp/multidoc/b/src/main/java/b/B.java'

See https://github.com/blindpirate/multi-doc for the example.

Interesting. I haven’t tried it yet, but can you elaborate on why you’re using the “whenTaskAdded” mechanism, as opposed to “tasks.withType(Javadoc)” as I had before?

tasks.withType(Javadoc) { } will apply the closure to all tasks with type Javadoc IMMEDIATELY. However, at this point, the aggregateJavadocs task gradle-aggregate-javadocs-plugin registered doesn’t exist yet (because it’s created in afterEvaluate here), so this wouldn’t take effect.

whenTaskAdded is different. The closure it registered would run every time when a new task is added, so we can configure aggregateJavadocs task once it is added by gradle-aggregate-javadocs-plugin.

Please let me know if there’s anything unclear.

That’s fine. I assumed it had something to do with timing of when things exist and don’t exist, just wanted that clarified.

The suggested solution works perfectly.

Thanks for the help.