Shadow plugin works incorrectly while depending on Nar packages

Help/Discuss
1

I tried to use the Shadow plugin, but it cant locate the dependencies correctly, which are produced by <packaging>nar</packaging> label with maven-nar

the exception stack is like

[2022-09-30 10:31:57,947] [pulsar-client-io-1-1] [org.apache.pulsar.client.impl.ProducerImpl] WARN [test] [standalone-0-11] error while create opSendMsg by batch message container
java.lang.NoClassDefFoundError: com/scurrilous/circe/checksum/Crc32cIntChecksum
	at org.apache.pulsar.common.protocol.Commands.serializeCommandSendWithSize(Commands.java:1519) ~[sample-pulsar-gradle-all.jar:?]
	at org.apache.pulsar.common.protocol.Commands.newSend(Commands.java:540) ~[sample-pulsar-gradle-all.jar:?]
	at org.apache.pulsar.common.protocol.Commands.newSend(Commands.java:507) ~[sample-pulsar-gradle-all.jar:?]
	at org.apache.pulsar.client.impl.ProducerImpl.sendMessage(ProducerImpl.java:771) ~[sample-pulsar-gradle-all.jar:?]
	at org.apache.pulsar.client.impl.BatchMessageContainerImpl.createOpSendMsg(BatchMessageContainerImpl.java:200) ~[sample-pulsar-gradle-all.jar:?]
	at org.apache.pulsar.client.impl.ProducerImpl.batchMessageAndSend(ProducerImpl.java:2002) ~[sample-pulsar-gradle-all.jar:?]
	at org.apache.pulsar.client.impl.ProducerImpl.lambda$connectionOpened$14(ProducerImpl.java:1631) ~[sample-pulsar-gradle-all.jar:?]
	at org.apache.pulsar.common.util.Runnables$CatchingAndLoggingRunnable.run(Runnables.java:54) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:176) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:394) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[sample-pulsar-gradle-all.jar:?]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[sample-pulsar-gradle-all.jar:?]
	at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: java.lang.ClassNotFoundException: com.scurrilous.circe.checksum.Crc32cIntChecksum
	... 18 more

and here is the sample build

Expected Behavior

better support for nar packages

Context (optional)

Placing this in build.gradle resolves the issue temporarily

// Workaround for invalid metadata for Bookkeeper dependencies which contain <packaging>nar</packaging> in pom.xml
configurations.all {
    resolutionStrategy.eachDependency { DependencyResolveDetails details ->
        if (details.requested.group == 'org.apache.bookkeeper' &&
                details.requested.name in ['circe-checksum', 'cpu-affinity', 'native-io']) {
            details.artifactSelection { ArtifactSelectionDetails asDetails ->
                asDetails.selectArtifact('jar', null, null)
            }
        }
    }
}

Steps to Reproduce

this issue is based on some github issue for bookkeeper project

the address is here

you can see the exact steps there for more details

Gradle version

7.x

2

Hi there.
If you seek for some fix / implementation, this is the wrong place.
This is a community forum where users like me help other users.

If you think this is a bug / shortcoming of the shadow plugin, report it to the issue tracker of the shadow plugin.

If you think this is a bug / shortcoming of Gradle itself, report it to the issue tracker of Gradle.

If you think (I have no idea what nar is) this is a bug in the metadata of those three bookkeeper libraries - i.e. wrong packaging configured -, report it to the issue tracker of bookkeeper.

As far as your work-around goes, to correct erroneous or problematic dependency metadata, better use a component metadata rule, for example like this:

dependencies {
    components {
        ['circe-checksum', 'cpu-affinity', 'native-io'].forEach { module ->
            withModule("org.apache.bookkeeper:$module") {
                allVariants {
                    withFiles {
                        addFile("$module-${id.version}.jar")
                    }
                }
            }
        }
    }
}
3

Thank you for your response.

I consider the nar package to be a custom packaging type. Here are some details nar-plugin.

In my current packaging setup, my project relies on Pulsar, which further depends on Bookkeeper. Meanwhile, I’m trying to incorporate the shadow-plugin into my own project. Therefore I’m afraid I may not be able to utilize the configuration you recommended.

Do you have any recommendations for this configuration?

4

Meanwhile, I’m trying to incorporate the shadow-plugin into my own project.

Which is an error in itself imho. :slight_smile:

Therefore I’m afraid I may not be able to utilize the configuration you recommended.

I don’t get what you mean.
What I showed does the same you did, just in a clean way and works perfectly fine in your example project.