FileTrees Union and matching() - How To?

Help/Discuss
1

Consider this file tree:

def myTree = fileTree('dir1/subA') + fileTree('dir2/subB')

I would expect the following to contain all files in dir2/subB/resources

def resources = myTree.matching(new PatternSet().include('dir2/subB/resources/*'))

But it doesn’t. From my understanding, when using matching() on a simple file tree, the patterns are interpreted relative to the tree root. This seems to carry into Composite Trees, but this creates ambiguity because 'resources/*' could match files in 'dir1/subA/resources' too.

Is there any solution to filter myTree while using complete patterns (so I won’t mix up trees)?

2

Let’s assume the following file structure

dir1/dir2/dir3/file.txt

You could find the file by the following

fileTree('dir1').matching {
   include 'dir2/dir3/file.txt'
}
// or
fileTree('dir1/dir2').matching {
   include 'dir3/file.txt'
}

Basically whatever you choose as the root is “lost” and all subsequent matching is relative to the file tree root. Not relative to the project root.

If you add two fileTrees together think of it as zipping two folders into the root of the same zipfile or copying the contents of two folders into the root of a file system

3

Thanks for the reply.

I see, it is as I feared. Then perhaps there is a solution to my root problem instead:

I have a text file whose contents are a list of file system paths, one per line. The paths may be relative to the project root (they don’t start with / or an equivalent) or absolute (they start with /). Let’s call this file searchPaths.txt.

The task is basically to copy all files in the directories listed to new directories depending on which pattern set they match (flattening any hierarchy, collisions are handled). If a file matches the first pattern, it’s not tested against the others. The DSL looks like this:

apply: my-vsp-plugin

vsp {
    files = buildFileTreeFromFile('searchPaths.txt')
    subdirs {
        md1 {
            pattern.include('d1/*.xml')
        }
        md2 {
            pattern.include('d2/*.yaml')
        }
        root {
            pattern.include('**/*')
        }
    }
}

I’m trying to do this with FileTree to leverage matching(). But I cannot simply create a FileTree rooted at the file system root because gradle tries to scan the whole file system when I do that (this fails due to permissions problems), even if using a filtered tree. But if I read each dir on searchPaths.txt as it’s own tree and combine they all I lose context when trying to filter the tree.

Is there a better approach to this?

4

A couple of options off the top of my head

  1. Copy everything into a temp directory, putting each line in a subfolder under the root. The subfolder retains information about the “search path”. Use the temp directory as a root for a FileTree for matching

  2. Use a combination of regex to match the search path and matching() to match files. Perhaps you can search through the gradle or ant sources and use one of the path matching utility classes rather than your own regex

5

Thanks for the suggestions, but after grappling a bit more with this I’ve concluded a FileTree isn’t really the best tool to achieve it.

One important point I haven’t realized before is that I can’t recurse on the paths listed in searchPaths.txt. So far that was happening as a side effect of the way I as filtering the tree; I was rooting it on the project root and adding include specific patterns for each line / pattern combination. So, if searchPaths.txt contents was dir1/resources, and the patterns where ['*.xml', '*.yaml'], my tree would be the equivalent of fileTree(project.rootDir).include('dir1/resources/*.xml', 'dir1/resources/.*yaml') and this tree would latter be filtered with matching(). This failed for absolute paths, hence the changes to Unions.

I’ve decided to tackle this with FileCollection instead, and try to filter it using a PatternSet; But that belongs in another thread.

Thanks again.

6

Perhaps you could have searchPaths.properties

d1=path/to/d1
d2=path/to/d2

From this you could create a Map<String, FileTree>. Then you could use the keys (d1, d2 etc) to get from the dsl to a FileTree

It’s hard to know exactly what you’re trying to achieve without seeing real sample values

7

While that would preserve the original source path info, it still would have the same problem of recursing when it should’t.

I’ve managed to solve my problem by:

  • Using listFiles(FileFilter) to get only normal files inside the paths
  • Adding those to a FileCollection
  • Defining a PatternSet to filter this collection later on.
  • During task execution I convert the PatternSet into a equivalent Spec<File> (with the help of PathMatcher, whose glob syntax is a superset of PatternSet includes/excludes) and filter the FileCollection with it.

The DSL ended up somehow OK:

vsp {
    def final sharedLib = readLibFile("$rootDir/LIBRARY_PATH")
    def final D7Lib = readLibFile("$rootDir/LIBRARY_PATH_D7")
    def final K3Lib = readLibFile("$rootDir/LIBRARY_PATH_K3")

    searchPaths {
        D7 {
            files = sharedLib + D7Lib
            virtualDirs {
                ZeosN2 { order = 1; location.set('Z'); filter.include("**/ZEOS*/src/*/*") }
                root { order = 2; location.set('.'); filter.include("**/*") }
            }
        }

        K3 {
            files = sharedLib + K3Lib
            virtualDirs {
                ZeosN2 { order = 1; location.set('Z'); filter.include("**/ZEOS*/src/*/*") }
                root { order = 2; location.set('.'); filter.include("**/*") }
            }
        }

        WithoutQt {
            files(sharedLib + K3Lib) {
                exclude('**/kylix/libQ/*')
            }
            virtualDirs {
                ZeosN2 { order = 1; location.set('Z'); filter.include("**/ZEOS*/src/*/*") }
                root { order = 2; location.set('.'); filter.include("**/*") }
            }
        }
    }
}

Thank you very much for your help.