Flatten found files to copy in resources

[Raja_Nagendra_Kumar1 (nagendra.raja)]

I have my resources in all over the source code… however all of them are in various sub directories with dir name res…

I wish test task picks all the files in res directory, where ever they are found in the root and copy them into resource build/resources/test/ folder as flattened file… (i.e. ignoring the source all the sub-directory info

instead of this

test {
	java {
	    srcDir("src/main/java")
	    include("**/tests/**")
	}

	resources {
	    srcDir("src/main/java/com/tejasoft/sboot/api/tests/archunit/res")
	}
    }

I want to write as

test {
	java {
	    srcDir("src/main/java")
	    include("**/tests/**")
	}

	resources {
	    srcDir("src/main/")
            include ("**/res/**")
	}
    }

[Raja_Nagendra_Kumar1 (nagendra.raja)]

Can any reply please… ant copy as simple attribute to say flatten=true…

https://ant.apache.org/manual/Tasks/copy.html

ant is so simple and too clear… hope has something equivalent

<copy flatten="true" todir="${test}/WEB-INF/lib" overwrite="${overwrite}">
                <fileset refid="buildJars"/>
                <fileset dir="lib">
                    <include name="commons-collections-*.jar"/>
                    <include name="commons-io-*.jar"/>              
                    <include name="kodo/*.jar"/>
                    <include name="mail*.jar"/>    
                    <include name="activation*.jar"/>               
                    <include name="guava*.jar"/>
                    <include name="**/*.jar"/>                          
                </fileset>          
            </copy>

I really don’t want to list absolute directory names… wish to use ** to find all directories by gradle itself

[justsomeone]

Hello @Raja_Nagendra_Kumar1

first i am new to grade world so i could miss something

i tried with the sourceSet i it did not work

as follow

java {
    sourceSets {
        main {

            resources {
                srcDirs {
                    fileTree(layout.projectDirectory.dir("src/main/java/resources")).getFiles()
                }

            }
        }
    }
}

but i tried it with copy task as follow and it work fine

tasks.register("copyResources",Copy){

    from fileTree(layout.projectDirectory.dir("src/main/java/resources")).files

    into layout.buildDirectory.dir("resources")

}

you can use that task and make other task depend on it till you get a better solution from more experience people

hope that help and have a nice day

[Raja_Nagendra_Kumar1 (nagendra.raja)]

Thank You @justsomeone, I did not want to give an absolute path as I refactor a lot, hence wish to use ** notation so that, any refactoring, I am void of the extra burden of changing the paths in build scripts too.

That way, in ant the solution is refactoring friendly…this way…

<copy flatten="true" todir="${test}/WEB-INF/lib" overwrite="${overwrite}">
                <fileset refid="buildJars"/>
                <fileset dir="lib">                
                    <include name="**/activation*.jar"/>               
                    <include name="**/guava/***.jar"/>
                    <include name="**/own/**/*.jar"/>                          
                </fileset>          
            </copy>

what flatten does it irrespective of jar file in various sub-directories of lib, it would still copy them in ${test}/WEB-INF/lib folder, I am looking for similar solution in gradle…

[justsomeone]

you welcome got you then hope someone help you soon also try with stackoverflow

[Raja_Nagendra_Kumar1 (nagendra.raja)]

nope… none are here to help… time to have our own tasks for this… gradle experts pl. share your expertise… on this simple and very trivial (as per ant style) need

[justsomeone]

@Raja_Nagendra_Kumar1

they a way to call ant from gradle not sure if that would help you or not

https://docs.gradle.org/current/userguide/ant.html

hope they show up soon and have a nice day

[Vampire (Björn Kautler)]

I didn’t test it, but something like this should flatten the files I think:

tasks.processResources {
    eachFile {
        relativePath = RelativePath(true, relativePath.lastName)
    }
}

[Raja_Nagendra_Kumar1 (nagendra.raja)]

found this solution with chatgpt help, it works

tasks.register('copyTestResources', Copy) {
        from(sourceSets.main.java.srcDirs) {
    	include '**/aop.xml'
    	eachFile {
    	    it.path = it.name
    	}
        }
        into 'build/resources/test/META-INF'
        includeEmptyDirs = false
    }

[Vampire (Björn Kautler)]

Which is the same I told you, just that mine is the more object oriented way, while ChatGPT’s is the more String-y way, but well, I don’t expect any correct or good answer by it anyway from what I’ve seen so far.

[Raja_Nagendra_Kumar1 (nagendra.raja)]

having kotlin version of this helps too

as it says sourceSets.main.java.srcDirs not found an issue with using ‘it’.

[Vampire (Björn Kautler)]

In Kotlin DSL accessors are lazy by default.
So sourceSets.main does not give you the source set but a provider holding the source set.
The IDE will greatly help you there with the Kotlin DSL too, showing types and proper code completion.
If you really need the value eagerly, just add a .get(), otherwise for example use .map { ... }.

[Raja_Nagendra_Kumar1 (nagendra.raja)]

sourceSets.getByName(“main”).java.srcDirs

this worked…not happy with such syntax though, maybe I need to forget gradle that may take some time…

at last this is the complete kotlin way…

tasks.register<Copy>("copyExtraTestResources0") {
    from(sourceSets.getByName("main").java.srcDirs.first()) {
	include("**/test/**/res/META-INF/services/**")
	eachFile {
	    path = name
	}
    }
    into("$buildDir/resources/test/META-INF/services")
    includeEmptyDirs = false
}

[Vampire (Björn Kautler)]

Don’t do that.
Never have multiple tasks with overlapping output directories.
That destroys up-to-date checks, it makes build cache impossible, it makes task output fingerprinting void.
It makes you need to define strange explicit dependencies to prevent ordering and dependency issues, …

Assuming those files are in src/test/resources, just configure the existing processTestResources task like I showed for the processResources before you said it is about test resources.

[Raja_Nagendra_Kumar1 (nagendra.raja)]

Hi Vampire,

The biggest challenge with the standard directory structure i.e both test folder and main folder at the same level, on large projects managing respective test files is becoming too complex, hence we always create test folder under the respective source code directly…

This does give us great benefit as developers to not keep moving from one directory to another especially as we practice DevTests - based development (a kind of TDD, but does not enforce test to be written first and also include as many kinds of tests be written by developers including integration, e2e, mock etc)

As this is not a standard directory structure we have to bit more work at the script level, but that is one time but having such a directory structure saves us so much intellectual energy. We also make sure respective resources also are close to the respective main or test code, so that, no confusion of changing them when the code changes.

I also note your feedback of up-to-date checks, I agree this may also need to consider to further, especially in IDE context. Thank You for brining up this point.

Regards
Raja Nagendra Kumar,
Code Doctor & C.T.O
nagendra.raja@tejasoft.com
+91-9886723872
http://www.linkedin.com/in/nagkumar

[Vampire (Björn Kautler)]

Sure, both approaches of code layout have their pros and cons.
I’m not sure why you explained it, I didn’t say anything against it.

But nevertheless, you must not have multiple tasks with overlapping outputs.
The processTestResources task and your copyExtraTestResources0 have overlapping output directories, as processTestResources has $buildDir/resources/test as output directory and that is particularly bad.

Usually - as said already - you should instead configure the existing task to include the files you want like

tasks.processTestResources {
    from(sourceSets.main.get().java.srcDirs) {
        include("**/test/**/res/META-INF/services/**")
        eachFile {
            path = "META-INF/services/$name"
        }
        includeEmptyDirs = false
    }
}

Btw. most often you should use Sync, not Copy to not have problems with stale files in the target directory.