Why do WARs have no dependencies with maven-publish

Hi,

we want to transition from the legacy deployment mechanism of the maven plugin, to maven-publish plugin. But we discovered that all war projects do not have any dependencies in the generated pom.xml at all.

Looking at gradles source code this is clear why as the web software component just returns emptyList:

But isn’t that a false behavior? war projects always had the same metadata as java projects why has this been changed? Should I open a Issue about that and is there a easy/suggested way to fix it?

Adding dependencies to other including projects is a workaround but transfers knowlege of the included wars requirements into other buildscripts, wich should not be a thing right?

kind regards
Daniel

Can you describe what you are trying to achieve? I can’t think of why you would want a war as a dependency. Perhaps you want to extract common classes and resources to a jar module which you depend on in other modules?

Unlike a jar, a war file has a WEB-INF/lib directory which contains all of the dependency jars. So the war file has everything you need to start the webapp. So it makes sense to me that a war has no dependencies.

I’m sure that you could use the pom.withXml {...} hook to add the dependencies to the pom but it’s likely best to refactor your build so that you depend on a jar instead of a war.

Hello @Lance,

I am working on a JEE project, where wars are just another packageable artifact, just like jars are.
Our final artifacts are all ears none of the wars we produce is delivered standalone.

We don’t include the runtime dependencies in the WEB-INF/lib directory as many other components (some jars, some wars) of the final ear will definitely use the same components too and will duplicate the artifacts at best.
What could be even worse is when the ear and war dependency-versions differ and without dependency informations the build won’t see/react to these conflicts.

So what we could do with the maven plugin is to declare the wars dependencies as providedCompile and consume these in the earlib configuration of the ear. This works very well while inside the wars origin multiproject build and with maven plugin in other referencing projects as well.

With maven-publish - so without the dependency informations of the war - other ears don’t have the informations wich artifacts in wich versions these web-modules want to use and no conflict resolution can kick in. Also we’ll have to duplicate the dependencies of the web-module in the consuming ear.

While I’ve read on probably more consistent approaches for stripped wars on the internet - the problem with all of them is the buildsystem (maven aint no better) support and even more problematic is the IDE integration.

Shoudn’t all artifacty carry their complete metadata? Even for a ear (while it is the last consuming unit in JEE) it seems useful to publish the complete dependency information it was built with to enable follow up tooling to consume it.

kind regards
Daniel

@geissld the metadata should only carry the information about what is needed to build/run the component. By default, the war packages the dependencies, so no external dependencies are needed. I think in general the current behavior is correct.

I am not sure I completely understand your use case. But if I read it correctly, you are using a “war” similar to a “jar”. I.e. you do not need/use the packaged dependencies, but you want the dependency information in the pom instead.

If that is correct, I would suggest to not use the war plugin, but instead use the java-library plugin and just change the extension of the jars to war. Something like:

plugins {
    id 'java-library'
    id 'maven-publish'
}

jar.archiveExtension.set("war")

publishing {
  publications {
    mavenJava(MavenPublication) {
      from components.java
      pom.packaging = "war"
    }
  }
}
1 Like

Well maybe a more expressive example is needed. Just for the record in Java EE the archive layout is like this:

Ear
\--- lib
 |   |- tool-jar-1
 |   |- tool-jar-2
 |- ejb-module (0..n)
 |- war-module (0..n)
    |-WEB-INF
      \--- lib
           |- tool-jar-3

Lets say I have a gradle build for a war:
build.gradle

plugins {
    id 'war'
    id 'maven-publish'
}

dependencies {
    // we do not declare these as implementation as they should not be packed inside the war
    providedCompile 'commons-logging:commons-logging:1.0'
    // still there are dependencies that noone else will or should use so we package them within the WEB-INF/lib
    implementation 'infrastructure:core:1.0'
}

publishing {
  publications {
    mavenJava(MavenPublication) {
      from components.web
    }
  }
}

Lets assume this war contains some useful functions (responding to infrastructure pings or general REST services) that must be included in every ear that is installed on a customer application server (in our case we never ever deploy wars self contained, they must always be part of a ear) and we deliver about 10+ different ears.

The ears are part of completely different projects (independend git-repositories, builds, developer, release-cycle).
These projects consume the war and package most of the wars dependencies (the provided ones) inside their ears lib directory.

plugins {
    id 'ear'
}
dependencies {
    // this works as usual
    deploy project(path: ':local-war', configuration: 'archives' )
    earlib project(path: ':local-war', configuration: 'providedCompile' )

    // this worked with 'maven' and is broken with 'maven-publish'
    deploy group: 'com.mycompany', name: 'common-war', version: '1.0'
    earlib group: 'com.mycompany', name: 'common-war', version: '1.0', configuration: 'provided'

    ...
}

This way a conflict resolution can occur if the local-war wants to use commons-logging:commons-logging:1.1 while the other expects 1.0. Even more important a conflict will occur when 2.0 is requested (in the application server normally the library in the ears lib wins, no matter what you package, so this information is very important!).

While this may not be a big thing in the minimal example, our ears have numerous wars included with many overlapping dependencies (I don’t exaggerate when the size of a ear without stripped wars would be 3-5 times bigger than it is now).

The war-labeled-jar can not work. The structure is totaly different (classes need to go in WEB-INF/classes, the web resouces to the root), IDE support will be completely broken and we will miss out libs that must be packaged.

Even if one does not like the use of the providedCompile configuration (or provided scope) for stripped wars there is still that big thing with dependency version conflicts.
A ear containing a incompatible version of a wars dependency in its lib directory will break the wars implementation and without dependency information no buildsystem has even remotely a chance to detect that.

So if I understand this correctly the ear plugin needs the dependencies, because it re-packages things. That is, the dependencies are stripped from the wars and instead the original dependencies should be used again, conflict resolved, and the packaged into the ear.

This sounds like there are indeed different variants/types of wars. The ones that are self contained and the ones that are stripped. This could maybe be modeled as different variants in the “war” plugin. Unfortunately, the “war” plugin implementation is a bit different as the “java-library” plugin and not as easily customizable (maybe that can be changed in the future when we have figured this one out). That’s why I suggested to use the “java-library” plugin. I just haven’t thought it through enough. :slight_smile:

But you can do this:

plugins {
    id 'java-library'
    id 'war'
    id 'maven-publish'
}

configurations {
    // remove the 'jar' artifact from the outgoing variants (jar task will not run anymore)
    apiElements.outgoing.artifacts.clear()
    runtimeElements.outgoing.artifacts.clear()
    // instead, add the 'war' artifact
    apiElements.outgoing.artifact(tasks.withType(War).first())
    runtimeElements.outgoing.artifact(tasks.withType(War).first())
}

publishing {
    publications {
        war(MavenPublication) {
            from(components.java)
        }
    }
}

So this will effectively still built you the war, because we tell the java library plugin to use the war instead of the jar task for building the artifact.

Now you can also customize the components.java component further, because it implements the AdhocComponentWithVariants interface which offers public API to customize the component:
https://docs.gradle.org/6.0-rc-1/userguide/publishing_customization.html#sec:adding-variants-to-existing-components

Would that work for you?

Hello jendrik,

thank you for the interesting insights. I’ll try to transform your suggestion into our companies war plugin wrapper (we provide prepared plugins that do a lot of default configurations without a single line of build.gradle dsl) and report back.
The documentation you linked is for Gradle 6.0 (we’re currently at 5.5.1) but the AdHocComponentWithVariants should already work, right? If I got it right then maybe the chapter Creating and publishing custom components may be even worth a try.

kind regards
Daniel

Hi Daniel,

Sounds good. Yes, most of it should work with 5.6.3 (and maybe 5.5.1) already. We just added better documentation for 6.0.

Also, “creating and publishing custom components” would be even cleaner for you I think. You just need to do that within a plugin to inject the factory. But if you already write your own plugin then you should definitely try it.
Also with the java component you have some things preconfigured that are bound to the java-library configuration/variants (implementation, api, runtimeOnly, …).
If you define your own component, you can set everything up from scratch using configurations as you please. You can also use the configurations provided by the war plugin to build up the new component. In that case, you might not even need the java-library plugin.
And if you end up with a working component setup, feel free to post here. We might be interested to promote that into the war plugin. :slight_smile:

Hello @jendrik,

I’ve had some closer look at the custom software component configuration for our plugin, That’s what I ended up with.

The easiest part was the configuration of the adhoc component:

    private void createSoftwareComponent(final Project project) {
        project.getConfigurations().getByName(WarPlugin.PROVIDED_COMPILE_CONFIGURATION_NAME).getArtifacts().add(artifact);
        // create an adhoc component
        AdhocComponentWithVariants component = softwareComponentFactory.adhoc(SOFTWARE_COMPONENT_NAME);
        // and register configuration variants for publication
        //component.addVariantsFromConfiguration(project.getConfigurations().getByName(WarPlugin.PROVIDED_COMPILE_CONFIGURATION_NAME), new JavaConfigurationVariantMapping("provided", false));
        //component.addVariantsFromConfiguration(project.getConfigurations().getByName(WarPlugin.PROVIDED_RUNTIME_CONFIGURATION_NAME), new JavaConfigurationVariantMapping("provided", false));
        component.addVariantsFromConfiguration(project.getConfigurations().getByName(WarPlugin.PROVIDED_COMPILE_CONFIGURATION_NAME), new JavaConfigurationVariantMapping("compile", false));
        component.addVariantsFromConfiguration(project.getConfigurations().getByName(WarPlugin.PROVIDED_RUNTIME_CONFIGURATION_NAME), new JavaConfigurationVariantMapping("runtime", false));
        // add it to the list of components that this project declares
        project.getComponents().add(component);
    }

Still there are two things that caught my attention. First the maven configuration provided cannot be targeted (see the commented out lines) as this throws the following error:
> Invalid Maven scope 'provided'. You must choose between 'compile' and 'runtime'

So I’ve switched to compile and runtime but I’m not sure this limitation will make everyone happy, as it also aktively prevents the generation of backward compatible pom files (the maven plugin produced these scopes).

With that in place the pom.xml actually looked promising but no artifact was deployed. First I simply added the artifact to the publication, but I guess that’s not how it is supposed to work.

    private void configurePublishing(Project project) {
        final PublishingExtension publishingExt = project.getExtensions().getByType(PublishingExtension.class);
        final MavenPublication mavenPublication = publishingExt.getPublications().maybeCreate(MyProjectUtils.PUBLICATION_NAME, MavenPublication.class);
        mavenPublication.from(project.getComponents().getByName(SOFTWARE_COMPONENT_NAME));
        mavenPublication.artifact((War)project.getTasks().getByName("war"));
    }

As I was looking at the JavaPlugin I found the configurations that I’ve never used in the dsl: apiElements and runtimeElements. Is there a explanation what they are for and how or better when one must use them?

Actually the JavaPlugin enhances these two configurations with the jarArtifact:

        PublishArtifact jarArtifact = new LazyPublishArtifact(jar);
        ...
        addJar(apiElementConfiguration, jarArtifact);
        addJar(runtimeConfiguration, jarArtifact);
        ...

    private void addJar(Configuration configuration, PublishArtifact jarArtifact) {
        ConfigurationPublications publications = configuration.getOutgoing();

        // Configure an implicit variant
        publications.getArtifacts().add(jarArtifact);
        publications.getAttributes().attribute(ArtifactAttributes.ARTIFACT_FORMAT, ArtifactTypeDefinition.JAR_TYPE);
    }

I was looking for something like that, but as I’am not registering the war task in my plugin I have no Provider<?> and can not initiate a LazyPublishArtifact. So I ended up implementing this:

    public void apply(...) {
        ...
        final ArchivePublishArtifact artifact = new ArchivePublishArtifact((War)project.getTasks().getByName(WarPlugin.WAR_TASK_NAME));
        addWar(providedCompileConfiguration, artifact);
        addWar(providedRuntimeConfiguration, artifact);
        ...
    }

    private void addWar(Configuration configuration, PublishArtifact jarArtifact) {
        ConfigurationPublications publications = configuration.getOutgoing();
        // Configure an implicit variant
        publications.getArtifacts().add(jarArtifact);
        publications.getAttributes().attribute(ArtifactAttributes.ARTIFACT_FORMAT, "war");
    }

To be honest I actually do not understand what the attributes imply, when they are used and why there’s a difference between the api and apiElements configuration or if that is another way to express what was formerly the artifacts configuration.

Also why are variants tied one-to-one with configurations. Adding a new variant when mapping a configuration to a maven scope fells kind of wrong, as in my simple case all configurations may be relevant for the same consumer/variant of my artifact.

I hope you have the time to answer some of the questions and my implementation is actually in line with the intentions of the software component idea - always love to understand a bit more about the gradle concepts to be able to implement better plugins.

kind regards
Daniel

Update:
I found this very interesting documentation: https://docs.gradle.org/6.0-rc-1/userguide/cross_project_publications.html#sec:simple-sharing-artifacts-between-projects
that realy helps to understand the concept much better.

So if I read this correctly the whole variant aware dependency resolution thing is only starting to gain momentum when the consuming Ear attaches attributes to the deploy configuration?
We’ll be able to declare deploy project(':local-war') dependencies when the war plugin provides a software component variant with attributes (lets say category=ear-module) and the same attributes are used on the Ears deploy configuration. This should then also be transfered to ejb jar projects.

That would realy make a interesting use case, but will also require a ejb plugin and changes to the war plugin and the ear plugin.
For the war that might result in a new earProvided configuration that will be used to declare dependencies for a software component variant that is consumed by a ear while a standalone variant must package these dependencyies in the war, right? I currently cannot imagine how that might look like when publishing those variants (or should they be different components?) to a maven repository.

Hi Daniel,

I’ll answer some questions and hope it helps you to move forward:

First the maven configuration provided cannot be targeted

The reason is that scope=provided has no meaning in a published POM. Neither Maven nor Gradle do anything with those dependencies. It is information for the local build only (like compileOnly in Gradle). It is often published in any case, because in Maven you basically “publish the build file”. So there is a lot of “useless” information in the published pom.
The maven plugin in Gradle was behaving in a similar “dumb” way. Just publishing all (or most) of the configurations you defined in your build.
But for completeness, we should maybe consider adding the option to map to provided. I think what you should do with the current mechanism is add a mapToOptional() because that also means that the dependencies are not used by Maven - which is what you want for the “default” war variant where everything is packaged.

Actually the JavaPlugin enhances these two configurations with the jarArtifact: …

Instead of configuration.getOutgoing().getArtifacts().add(...), which only accepts Artifacts, you can use configuration.getOutgoing().artifact(tasks.named("war")) to wire the output of an archive task as artifact you want to publish. Sorry the API is a bit messy.

So if I read this correctly the whole variant aware dependency resolution thing is only starting to gain momentum when the consuming Ear attaches attributes to the deploy configuration?

Thats correct. Basically you define dependencies between components (or projects). Only when you resolve the dependencies in a certain context, variants are selected. So Gradle can ask things like:
“give me everything I need to compile this library for Java 6” or “give me everything I need to run this application with Java 9” . And the actually selected variants (and their artifacts) can differ.
So a (EAR) Plugin can say I want all “stripped-wars-with-dependencies” and the it will get them for all dependencies with the dependencies they declare. Also for the ones further down the graph.

We also just recorded a Webinar on the topic. Maybe that also brings some more understanding:

If you have some working code, were you have more additional detailed questions, I think it would be easier if you could share that. Maybe as a small example on GitHub or something like that.

FYI - https://github.com/gradle/gradle/issues/11254

Thank you very much for the comprehensive reply. I’ll definitely watch the webinar, but for now the code from above is all I have and unfortunately it’s sufficient for my current usecase (you know there is only so much you can do in the time you have).

I think to have it done right one will have to enhance the core plugins and thus improve the JEE build model. I’ll definitely post back when I’ve had time to implement something.
Once again thank you for the explanations so far.