Main vs. test, compile vs runtime classpaths in Eclipse - once and for all, how?

Help/Discuss
1

This Eclipse integration problem has been causing us issues for years now but we were able to somehow live with the itch. With some changes to the projects this is now becoming significantly harder. While Gradle does differentiate between “main compile”, “main runtime”, “test compile” and “test runtime” classpaths, as noted in numerous places and forums, Eclipse does not, at least not to the same extent. Eclipse projects only have one classpath and all of it is used for compilation. It does not differentiate between main and test code either - it is all just code to it. Runtime classpaths are supposed to be defined by run/launch configurations and tests are supposed to be in separate projects.

As far as I know and have experienced, Gradle’s Eclipse plugin behaves like most tools out there - it kinds of gives up on Eclipse and rolls all relevant Gradle classpaths (via runtime and testRuntime that extend compile and testCompile) into a single Eclipse project classpath for ‘that’ project. Tests are declared as an extra source tree for the same project. Normally this is of little issue, as naughty main code referencing test code would fail in a gradle build even though it would pass in the Eclipse build and projects may not be large/complex/managed enough to warrant preventing access to some code that is supposed to be a runtime-only dependency.

Both of these problems become nasty in certain situations. We have in-Eclipse code generation tools (Gradle equivalents we use are neither the problem nor the solution) that get messed up by visibility of test code as if it were project code in Eclipse - they fail. As for compile vs. runtime dependencies we have strict rules as to which libraries our own code can use directly (compile classpath) and which it should not because we want to minimize the number of dependencies and preserve the ability to move to a different library (mainly transient, runtime-only dependencies).

To this extent I attempted to create a gradle plugin that will create separate Eclipse test projects ‘out of thin air’ based on the main projects configuration (these would not be in source control, as they only exist to help with Eclipse model differences). This I more-or-less succeeded. In creating the runtime classpaths, so far, I failed miserably.

Again, for Eclipse, runtime classpaths should be defined in launch configurations. However, to do this I would have to generate those just like the eclipse plugin generates the .classpath files. Since developers often copy the launch configs to make local tweaks, their copies would end up being disconnected from this generation and would not receive any classpath updates. So I abandoned that idea.

I tried using the same trick I did for the test projects - creating helper ‘runtime classpath’ projects out of thin air, with no code, only dependencies (referenced libraries). Not only I hit some weird issues with this (due to either my own lack of understanding of how Gradle works or Gradle bugs, I may write about these separately), I re-discovered that this very quickly leads to multiple occurrences of the same library in the runtime classpath - one per referenced project that brings it. Eclipse does not eliminate duplicates, it is happy to have them. With startup classpath scanning code (say searching for beans, annotations, etc) this becomes a serious issue - that of performance and can fail when the same class is apparently found multiple times.

The ongoing plan is now to abandon the “runtime classpath only projects” and instead replace them with dynamically (gradle) generated “Eclipse Libraries”. It is then the library (or libraries) that would be included in the launch configuration classpath instead of ANY non-project dependencies (projects would either not export their dependencies or these would have to be ignored).

Before I start doing this, is there any existing work done on this and/or any recommendations?

Thanks!

1 Like
2

You can right click on a class/method and choose “Run As” --> “Gradle Test”. AFAIK this runs the test via the Gradle Daemon using the exact same classpath as building from command line. An extra benefit of running your tests this way is that any doFirst and doLast blocks will also be executed. I found that the output of running this way was less pretty/clickable than the normal "Run As --> “JUnit Test” so I find myself using the Gradle Test runner for test cases known to be broken in Eclipse and the JUnit runner for everything else.

More info here

3

That is great to know, but it solves the problem I don’t have yet. I can run tests when I can find them. To find a particular test to run, I must have it listed somewhere. Normally that is just another source tree of the same project as the main code. In that case the main code also has access to test code (in Eclipse) and to testCompile classpath (at the very least), both of which are what we need to get rid of.

4

Your description is very “wordy” and it’s not entirely clear what your problem is.

Is your issue that “naugthy” code compiles in eclipse and fails in the gradle build? Or is it a runtime issue running a main method or a test case in Eclipse where the classpath is different to Gradle? Or something else?

Please give a concrete example which demonstrates the issue(s) you are facing including interesting code snippets and the paths of the related files

5

Well, yes. I apologize for the “wordiness” but I needed to set the proper context in one place. Yes, in a way, you could look at it as that the problem is that “naughty” code compiles in Eclipse. It is not the problem that it fails in Gradle - it should.

The problem is the entire model of (lack of) mapping between Gradle classpaths and Eclipse ones, while maintaining separation. I am looking for a complete solution, not a partial one. This means:

  1. The main code should not have visibility into test code. The test code should have visibility into main code. [For a single project].

  2. The main code should only be able to directly reference “compile” dependencies, not those that are “runtime only” (in runtime config but not compile).

  3. Analogous to (2), the test code should only be able to directly references “testCompile” dependencies (and main code)… Generally speaking “testCompile” configurations extends the “compile” one.

  4. Since (2) and (3) do not contain (test)Runtime dependencies, these need to be put somewhere so that we have the chance of running the main code and tests from Eclipse. Generally speaking “runtime” configuration extends “compile” and “testRuntime” extends BOTH “runtime” and “testCompile”.

  5. When we want to refresh dependencies (e.g. gradle --refresh-dependencies eclipseClasspath) this should update compile, testCompile, runtime and testRuntime classpaths in Eclipse, wherever they are, so that Eclipse launch configurations (continue to) work. This is for both main code and test launch configurations.

We are struggling to achieve the above and are looking for a whole solution for this.

6

Unfortunately Eclipse doesn’t support multiple classpaths in a single project which is why both Buildship and the Maven plugin do the same fudging and join all the classpaths together. As with all lossy conversions there is detail that is not available in Eclipse that is available in Gradle (and Maven)

If you wanted a boundary that is respected by both Gradle and Eclipse you could move all test code to separate projects and only include what’s needed. Eg

  • api
  • api-test (depends on api)
  • server (depends on api)
  • server-test (depends on api-test & api)
7

Yes, I described doing that. We separate API, implementation, main vs. test across both of these. I made a Gradle plugin that creates extra Eclipse test projects for each of these automatically. So I can deal with the compile classpath this way. However, that work is not 100% complete yet. Before I spend any more time on it I would like to know if there is something similar already in existence that I could not find myself.

Furthermore, this does not address the runtime (and testRuntime) classpaths. I noted what I tried to do with these and what I seem to have to do. Again, looking for existing solutions before I try my own.

A bit of info re my handling of tests - it is a borderline (if not entirelya) hack in part. What it does is:

  1. Configure eclipse “plusConfigurations” to be “compile” only. It also automatically configures “minusConfigurations” to exclude everything that is in “compile” configurations of dependency projects (inherited from them).

  2. If any Eclipse-related task is requested, it omits the test source code from the source sets of all projects. Instead I configure a project named after the original with the suffix “.test” added and make its source set contain the test source tree from the main project. To have this work in Eclipse I have to also configure a linked folder in Eclipse. I don’t like this but I have no better way - I do NOT want to have separate test projects in the source repository - I want default Gradle behaviour outside Eclipse.

  3. I set plusConfigurations on those *.test projects to be the testCompile of the main project. I actually tried creating a *.test project specific configuration and have it reference the testCompile of the main project but this did not work. I equivalently set minusConfigurations to exclude inherited stuff.

  4. I continue relying on a gradle plugin we made before which enhances the way project dependencies are declared. Specifically, it allows us in a single line to declare a non-transitive compile-time dependency on a library such that its own transitive dependencies are accounted for only in the runtime dependency. There is more to this, but you can think of it, in simplest terms, as making “compile” configuration not transitive (“runtime” remains transitive) and define each compile-time library dependency twice, i.e. instead of:

compile someLibrary

the effect is the same as:

compile someLibrary
runtime someLibrary

This prevents leaking transitive dependencies that are not specifically called for as our dependencies into our compile time classpath. Another enhancement is that when a dependency against a project is declared, we do not default to the ‘default’ configuration but match each. If project A depends on project B that means that A’s compile depends on B’s compile, A’s runtime depends on B’s runtime, A’s testCompile depends on B’s testCompile and A’s testRuntime depends on B’s testRuntime.

Note - if Eclipse-related tasks are not called for, the “virtual” *.test projects are not created in settings.gradle either. Since this is too early for plugins, I am forced to reimplement a bit of logic in it to do this.

8

Hey guys,

indeed, making Eclipse project handling more fine grained is currently not solved by Gradle. However, we are aware of these problems and intend to address them in the future. More specifically, when we do the Eclipse integration for the new Software Model.

In the Software Model, a project is split into components. So you might have a Java library component and a test component for instance. Our basic approach will be creating one Eclipse project for each component, which will separate the different classpaths.

The compile vs. runtime problem can only be solved elegantly with an Eclipse plugin. There are extension points that allow you to say things like “This compile classpath entry should not be visible at runtime” and “this classpath entry should be added to run configurations, but not to the compile classpath”. This is something we want to improve in Buildship, the Eclipse integration for Gradle.

Neither of these two projects has a high priority at the moment, so if you need this solved right now, then rolling your own is your best bet.

Cheers,
Stefan

1 Like
9

Thanks for responding. I am somewhat cautious about requiring more plugins in Eclipse. They would certainly be helpful but it would be awesome if we could do a bit more without it. What I am trying to do now is to generate Eclipse libraries (library containers) with runtime entries in them. Presently facing the issue where the runtime configurations also include gradle-built project jars, so I have to create more configurations that don’t have these, as project code needs to come from Eclipse. So more support for dealing with existing Eclipse concepts (libraries, workspace setup, etc) would also be very helpful.

Thanks again!

10

This would be partially resolved by Add the “test” classpath attribute to test sources and dependencies #689