Merge duplicate props files

Help/Discuss
1

gradle merge duplicate files

I have this file META-INF\services\org.junit.jupiter.api.extension.Extension in multiple places in source each with respective properties related to that code module

However during the building all these property files needs to be merged and only file with the name org.junit.jupiter.api.extension.Extension should be copied

Is there a merge strategy if no by default, can I extend gradle some way and use it

2

Sounds like you try to build a bad-practice fat jar.
Such service files are one of the many reasons they are a bad idea.
See https://fatjar.net/ for more quotes about it.
If you really want to build such a mess you should probably use John Engelman’s shadow plugin which mitigates some of the problems these fat jars produce, amongst them merging service descriptor files.

3

Does fatjar help in mergining same file name properties that exist in different locations?

4

I want to something similar to said above, but looking for a duplicate strategy as Merge, so that, I get all the copy advantages.

How do I create a custom MERGE duplicate strategy

potentially a plugin in like this

https://remal.gitlab.io/gradle-plugins/plugins/name.remal.merge-resources/

I tried using this… but it is not working…

5

Does fatjar help in mergining same file name properties that exist in different locations?

I assumed you do build a fatjar and because of that have the problem with the same-named files where only one survives.
A fatjar imho does never help for anything but only produce problems.
I would never recommend building a fatjar, except maybe a good-practice one like Spring Boot is doing.

How do I create a custom MERGE duplicate strategy

I don’t think you can. At least I’m not aware of.
Something like that plugin might be what you need then.
If it does not work, I suggest you open a bug report to it.

1 Like
6

Thank You Vampire, found this workaround (based on some insights from #ChatGPT), it does work in my context,

 register<Copy>("copyMetaInfoTestRes") {
	from(sourceSets.main.get().java.srcDirs) {
	    include("$TEST_DIRS_PATTERN/res/META-INF/services/**")
	    `checkDup&Append`()
	}
	into("$buildDir/resources/test/META-INF/services")
	includeEmptyDirs = false
	dependsOn("copyJU5Props")
	doLast {
	    doCheck("$buildDir/resources/test/META-INF/services/org.junit.jupiter.api.extension.Extension",
		    "com.tejasoft.tests.ju.utils.scane.aop.exts.PrintAOPCount")
	}
    }

and

fun Copy.`checkDup&Append`()
{
    eachFile {
	path = name
	val existingTargetFile = File(destinationDir, name)
	if (existingTargetFile.exists())
	{
	    val duplicatedFileContent = file.readBytes()
	    Files.write(existingTargetFile.toPath(), duplicatedFileContent, StandardOpenOption.APPEND)
	    exclude()
	}
    }
}
7

Again, do not trust any of the non-sense ChatGPT produces.
I already explained that in other threads, also by you.
Also, do never, ever, ever, ever have tasks with overlapping outputs.
With your task you write into the output directory of the processTestResources task and thus have overlapping outputs.
This causes a lot of problems and performance degradation, so never, ever, ever do that.

1 Like
8

I seem to have run into the same problem, but I’m not trying to create a fatjar, and the original poster has not confirmed that they are trying to create a fatjar either (I don’t know why someone would want Gradle to create a fatjar for JUnit tests, so it seems unlikely to me).

I have a project which has “META-INF/services/org.junit.jupiter.api.extension.Extension” in a java-test-fixtures project, and a JUnit 5 project that depends on the test fixtures project.

I’m simply running the :test task on the JUnit 5 project.

If the service file is only in the test fixture project, the JUnit project finds it.

If I add a service file into the JUnit project itself, the JUnit project only finds its own service file, and no longer finds the service file from the test fixture project.

I don’t know if this is a problem with JUnit, or the :test task in Gradle, or my project configuration, but the solution in this thread has issues as you stated and I don’t know what the proper solution would be.

9

Can you knit an MCVE? Then I might find some minutes to have a look at it. :slight_smile:

10

Apologies for the false alarm, it seems JUnit does find the services file, but it registers the extensions from those services in the wrong order. Even though the main project depends on the test fixtures project, the test fixtures extensions are executed after the main project’s extensions. I confirmed this while trying to make the minimal example.

The main project’s extensions rely on the test fixtures extensions setting some things up, so when JUnit ran the main project’s extensions first, it failed before it even had a chance to run the test fixtures extensions.

11

That’s then probably more a question to JUnit how to ensure order of extensions if it is possible at all.