Resource filtering in gradle eclipse buildship

I’m converting a maven project with has resource filtering (substitution of ${properties} in config files) also working inside eclipse. I just save a property file and the builtin maven plugin (m2e) saves the processed file in the target folder.

I’m now converting this project to gradle and used the “filter ReplaceTokens”. It works in the command like when I do “gradle assemble” and saves the processed file to the build directory.

But inside eclipse the imported gradle project just uses a “bin” directory and there the file is not processed when saved.

May this be an incorrect use of gradle/buildship by me or that feature as maven does is not supported in buildship? If it’s supported, can you point me to a small project doing this?

Hey Adriano,

this is not yet supported by Buildship. We would need task execution on auto build for that. That’s high on our priority list. If you are interested in contributing that feature, I’d be happy to walk you through the design process.

Cheers,
Stefan

Hi,

Same context: moving from Maven to Gradle, and on Eclipse side from m2e to BuildShip.

I’m interested to see proper resource filtering in Buildship. I can maybe help, since this “high priority” topic doesn’t seem to have been addressed after more than a year :slight_smile:

What is the best place to discuss this? Is there a JIRA / GitHub ticket somewhere?

Thanks

Julien

Hi,

The current state of affair is that the eclipse Gradle plugin can configure resource filters, but this information is not available via the EclipseProject model in the Tooling API. To implement the story one has to extend EclipseProject and consume the new information in the Buildship synchronization model.

You can follow up on this issue at eclipse/buildship#271 and on the linked stories. You can contribute to Gradle/Buildship via pull requests, we gladly guide you through the process.

Hi,

I have the impressions that we put 2 different things under the “resource filtering” concept.

  1. src/main/resources folder contains files that should be available on the classpath (to allow finding them with myClass.getResources()), but that should not be considered as source code. For example Eclipse should not try to compile .java files under src/test/resources, and also probably not validate all .js files in node_modules. The problem is that in Eclipse, there is no concept of “resource” folder. So there is a need to tell Eclipse to “filter” some files from being processed by the auto-build.

  2. variable substitution during resource files processing. This is called “filtering” in Maven, and I think this is why this is confusing. In Gradle we are using something like:

    processResources {
      filter ReplaceTokens, tokens: [
        'project.buildVersion': xxxxx
      ]
    }
    

In Eclipse, we wants to ensure that at runtime (like when running JUnit tests) the classpath contains the “resolved” resources, not the original ones.

I remember m2e spent a lot of time trying to find a working solution, that would not trigger recursive auto-builds.

When importing a Maven project into Eclipse with m2e, resource folders have all their content excluded:
<classpathentry excluding="**" kind="src" output="target/classes" path="src/main/resources">

It means Eclipse will not touch resources files at all. I guess the Maven Builder will take care to copy resources to the output folder, applying variable substitution if needed. I think the Gradle Builder could do the same.

Maybe there’s a solution for your use case. You can adjust the Eclipse project generation in the build script. You can define the same excluding pattern:

apply plugin: 'eclipse'

eclipse {
    classpath {
        file {
            whenMerged {
                def src = entries.find { it.path == 'src/main/resources' }
                src.excludes = ['**']
            }
        }
    }
}

Or, you can transform the sources folder to a directory inside the Gradle classpath container:

import org.gradle.plugins.ide.eclipse.model.Library
apply plugin: 'eclipse'

eclipse {
    classpath {
        file {
            whenMerged {
                entries.removeAll { it.path == 'src/main/resources' }
                entries += new Library(fileReference(file('src/main/resources')))
            }
        }
    }
}

That’s interesting options, but if I’m correct, none is 100% correct.

Option 1 (adding ‘**’ exclusion) alone will prevent files from being copied to the output folder. And consequently, running unit tests from Eclipse will not “see” them.

Option 2 doesn’t work because libraries are not exported to downstream projects. Using the Eclipse UI, I found that adding a class folder (+export it) is likely what is the more appropriate behavior.

Still the 2 options doesn’t support variable interpolation…

This may work if the source folder itself is added to the runtime classpath. Sure, it will also expose *.java source code files to the runtime in that case, so not entirely correct but this may be acceptable to at least some projects. Still investigating due to The project was not built due to "Resource already exists on disk" errors in eclipse

:frowning_face: We don’t use (or want to use) separate *.java code and resource folders but I tried something similar to above to set up the excludes on the main source directory. I was able to configure Gradle Eclipse plugin to set them in the .classpath files it generates:

        eclipse {
            classpath {
                file {
                    whenMerged {
                        entries.each { entry ->
                            if (entry instanceof  org.gradle.plugins.ide.eclipse.model.SourceFolder) {
                                entry.excludes += ['**']
                            }
                        }
                    }
                }
            }
        }

(Above is verbose because I had some trouble filtering the collection otherwise)…

But none of this takes effect through Buildship. What to do?

Edit 1: https://github.com/eclipse/buildship/issues/271 is still open. :frowning_face:

Edit 2: Even if I generate .classpath files before importing projects (the same way Buildship would create them) they get recreated when the projects are imported … without filters. :frowning_face:

A while ago I wrote a custom Buildship extension (mentioned here) that provides this feature. Could you please give it a go?
https://github.com/donat/buildship-sample-resource-filters/tree/master/deploy
To do that you need to:

  • clone the repository
  • install plugin: click Help > install new software > Add... > Local and select the deploy folder from the cloned repository.

I saw it and looked at it, actually. Thank you for your effort but we probably won’t go with this:

  1. I was mainly looking at this due to “The project was not built due to "Resource already exists on disk" errors in eclipse” but have since found what appears to be the near-root cause, if not a root cause. Will reply about that to that thread.
  2. We are using automated development setup environment based on https://plugins.gradle.org/plugin/com.diffplug.gradle.oomph.ide. I’d need to set up an actual p2 repository with your plugin and I don’t know how to do that yet.
  3. This will be deployed to many developers so hiccups aren’t good yet. I was willing to contribute but there isn’t much time - I have to deploy some sort of sturdy development environment solution soon.

We are using automated development setup environment based on https://plugins.gradle.org/plugin/com.diffplug.gradle.oomph.ide.

So you are using goomph. Good to know.

I’d need to set up an actual p2 repository with your plugin and I don’t know how to do that yet.

The deploy folder is a p2 repository. You can host it in any (probably internal) web server and it’s done.

This will be deployed to many developers so hiccups aren’t good yet. I was willing to contribute but there isn’t much time

Well, I get it and it’s a same on my side. This should be a Buildship feature, but my official schedule doesn’t allow me to work on this feature. This is why I implemented this in my free time. It may not be “Buildship-quality” (e.g. no cross-version testing and else), but it’s pretty damn close. If I get feedback on it I’ll integrate it into Buildship.

Yes, we are using Goomph. Pretty awesome, with few notes:

  1. We can’t use it to automatically import Gradle/Buildship projects. That is the only manual step left. It can import them but not as Gradle/Buildship … only as existing projects for which the eclipse plugin generates .project and .classpath files.
  2. Insufficient control over where the workspace is created, not even an ability to programatically determine where it is.

I do think that your efforts should be a part of Buildship, though.

If you are interested, there’s support for importing Gradle/Buildship projects into Eclipse with Oomph:https://github.com/eclipse/buildship/tree/master/samples/oomph-setup

I do think that your efforts should be a part of Buildship, though.

Eventually, it will, but I need feedback (if it works as other people expect it to work) before I want to invest more time into code coverage, etc.

Hmmm… sounds interesting. Is that instead or in addition to Goomph? I could do either, just would like to know.

It’s independent from goomph.

I may have to come back to your plugin. I seem only to have increased my chances of good luck but haven’t eliminated the failures altogether. This is crazy.

I am trying this route now. However, it seems that this configures exclusion of directories and not files (resources ) matching a pattern from within directories that should remain. I hope I just don’t know how to use this… Can you help me?

Having failed that, I wanted to try modifying Buildship itself… Failed, need help. Command-line build failed due to:

* What went wrong:

Execution failed for task ':assembleTargetPlatform'.

> Process 'command '/Users/.../.tooling/eclipse/targetPlatforms/eclipse-sdk/eclipse/Eclipse.app/Contents/MacOS/eclipse'' finished with non-zero exit value 13

Then I tried the official guide (https://github.com/eclipse/buildship/blob/master/docs/development/Setup.md) and failed again
after step 3 (before 4) using newest Oomph:

Calculating requirements and dependencies.
Cannot complete the request.  Generating details.
ERROR: org.eclipse.equinox.p2.director code=10054 Cannot complete the install because of a conflicting dependency.
  at org.eclipse.oomph.util.OomphPlugin.coreException(OomphPlugin.java:280)
  at org.eclipse.oomph.p2.internal.core.ProfileTransactionImpl.resolve(ProfileTransactionImpl.java:427)
  at org.eclipse.oomph.p2.internal.core.ProfileTransactionImpl.commit(ProfileTransactionImpl.java:339)
  at org.eclipse.oomph.setup.p2.impl.P2TaskImpl.perform(P2TaskImpl.java:903)
  at org.eclipse.oomph.setup.internal.core.SetupTaskPerformer.doPerformNeededSetupTasks(SetupTaskPerformer.java:3827)
  at org.eclipse.oomph.setup.internal.core.SetupTaskPerformer.performNeededSetupTasks(SetupTaskPerformer.java:3755)
  at org.eclipse.oomph.setup.internal.core.SetupTaskPerformer.performTriggeredSetupTasks(SetupTaskPerformer.java:3736)
  at org.eclipse.oomph.setup.internal.core.SetupTaskPerformer.perform(SetupTaskPerformer.java:3629)
  at org.eclipse.oomph.setup.ui.wizards.ProgressPage$9.run(ProgressPage.java:585)
  at org.eclipse.oomph.setup.ui.wizards.ProgressPage$11$1.run(ProgressPage.java:711)
  at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
  ERROR: org.eclipse.equinox.p2.director code=0 Software being installed: artificial_root 1.0.0.v1582660676379
  ERROR: org.eclipse.equinox.p2.director code=1 Only one of the following can be installed at once: 
    ERROR: org.eclipse.equinox.p2.director code=0 Java Development Tools Core 3.19.0.v201912311228-e1909-RELEASE (org.eclipse.jdt.core 3.19.0.v201912311228-e1909-RELEASE)
    ERROR: org.eclipse.equinox.p2.director code=0 Object Teams Development Tooling Core 3.20.0.OTDT_r276_201912111741 (org.eclipse.jdt.core 3.20.0.OTDT_r276_201912111741)
    ERROR: org.eclipse.equinox.p2.director code=0 Java Development Tools Core 3.20.0.v20191203-2131 (org.eclipse.jdt.core 3.20.0.v20191203-2131)
  ERROR: org.eclipse.equinox.p2.director code=1 Cannot satisfy dependency:
    ERROR: org.eclipse.equinox.p2.director code=0 From: artificial_root 1.0.0.v1582660676379
    ERROR: org.eclipse.equinox.p2.director code=0 To: org.eclipse.equinox.p2.iu; org.codehaus.groovy.eclipse.feature.feature.group 0.0.0
  ERROR: org.eclipse.equinox.p2.director code=1 Cannot satisfy dependency:
    ERROR: org.eclipse.equinox.p2.director code=0 From: artificial_root 1.0.0.v1582660676379
    ERROR: org.eclipse.equinox.p2.director code=0 To: org.eclipse.equinox.p2.iu; org.eclipse.jdt.feature.group 0.0.0
  ERROR: org.eclipse.equinox.p2.director code=1 Cannot satisfy dependency:
    ERROR: org.eclipse.equinox.p2.director code=0 From: Groovy-Eclipse Compilerless feature 3.6.0.v201912311228-e1909-RELEASE (org.codehaus.groovy.compilerless.feature.feature.group 3.6.0.v201912311228-e1909-RELEASE)
    ERROR: org.eclipse.equinox.p2.director code=0 To: org.eclipse.equinox.p2.iu; org.codehaus.groovy.headless.feature.feature.group [3.6.0.v201912311228-e1909-RELEASE,3.6.0.v201912311228-e1909-RELEASE]
  ERROR: org.eclipse.equinox.p2.director code=1 Cannot satisfy dependency:
    ERROR: org.eclipse.equinox.p2.director code=0 From: Eclipse Groovy Development Tools 3.6.0.v201912311228-e1909-RELEASE (org.codehaus.groovy.eclipse.feature.feature.group 3.6.0.v201912311228-e1909-RELEASE)
    ERROR: org.eclipse.equinox.p2.director code=0 To: org.eclipse.equinox.p2.iu; org.codehaus.groovy.compilerless.feature.feature.group [3.6.0.v201912311228-e1909-RELEASE,3.6.0.v201912311228-e1909-RELEASE]
  ERROR: org.eclipse.equinox.p2.director code=1 Cannot satisfy dependency:
    ERROR: org.eclipse.equinox.p2.director code=0 From: Groovy-Eclipse Headless feature 3.6.0.v201912311228-e1909-RELEASE (org.codehaus.groovy.headless.feature.feature.group 3.6.0.v201912311228-e1909-RELEASE)
    ERROR: org.eclipse.equinox.p2.director code=0 To: org.eclipse.equinox.p2.iu; org.codehaus.groovy.jdt.patch.feature.group [3.6.0.v201912311228-e1909-RELEASE,3.6.0.v201912311228-e1909-RELEASE]
  ERROR: org.eclipse.equinox.p2.director code=1 Cannot satisfy dependency:
    ERROR: org.eclipse.equinox.p2.director code=0 From: Eclipse Java Development Tools 3.18.200.v20191210-0610 (org.eclipse.jdt.feature.group 3.18.200.v20191210-0610)
    ERROR: org.eclipse.equinox.p2.director code=0 To: org.eclipse.equinox.p2.iu; org.eclipse.jdt.ui [3.20.0.v20191203-1301,3.20.0.v20191203-1301]
  ERROR: org.eclipse.equinox.p2.director code=1 Cannot satisfy dependency:
    ERROR: org.eclipse.equinox.p2.director code=0 From: Java Development Tools UI 3.20.0.v20191203-1301 (org.eclipse.jdt.ui 3.20.0.v20191203-1301)
    ERROR: org.eclipse.equinox.p2.director code=0 To: osgi.bundle; org.eclipse.jdt.core [3.20.0,4.0.0)
  ERROR: org.eclipse.equinox.p2.director code=1 Cannot satisfy dependency:
    ERROR: org.eclipse.equinox.p2.director code=0 From Patch: org.codehaus.groovy.jdt.patch.feature.group 3.6.0.v201912311228-e1909-RELEASE Eclipse Java Development Tools 3.18.200.v20191210-0610 (org.eclipse.jdt.feature.group 3.18.200.v20191210-0610)
    ERROR: org.eclipse.equinox.p2.director code=0 To: org.eclipse.equinox.p2.iu; org.eclipse.jdt.core [3.19.0.v201912311228-e1909-RELEASE,3.19.0.v201912311228-e1909-RELEASE]

Took 1 seconds.
There are failed tasks.

I see that there is a model class EclipseSourceDirectory and there is Buildship code (SourceFolderUpdater):

private IClasspathEntry toClasspathEntry(EclipseSourceDirectory sourceFolder, IClasspathEntry existingEntry) {
    SourceFolderEntryBuilder builder = new SourceFolderEntryBuilder(this.project, existingEntry.getPath());
    builder.setOutput(existingEntry.getOutputLocation());
    builder.setAttributes(existingEntry.getExtraAttributes());
    builder.setIncludes(existingEntry.getInclusionPatterns());
    builder.setExcludes(existingEntry.getExclusionPatterns());
    synchronizeAttributesFromModel(builder, sourceFolder);
    return builder.build();
}

… that seems to pass on the filtering patterns and eventually leads to:

    public IClasspathEntry build() {
        return JavaCore.newSourceEntry(this.path, this.includes, this.excludes, this.output, this.attributes);
    }

And JavaCore is JDT… so why doesn’t this work?