Improving the Tooling API

Old Forum Archive
1

I have written a Gradle plugin for NetBeans based on what Geertjan Wielenga wrote. The project can be found here https://github.com/kelemen/netbeans-gradle-project where the latest compiled binary (nbm) can also be donwloaded. However, I think I have reached the limit of the Tooling API of Gradle. These are the main limitations of the Tooling API (for me):

  • The Tooling API is very specialized and there can be many things in the

Gradle script which is not accessible through any model. - The performance is relatively poor because, the API returns everything

in a single call (including sources of dependencies). - Once an operation (parsing a model, executing a task, etc.) has been

started, it cannot be canceled. That is, I don’t see any way to do it.

Changes I would like ====================

Lower level Tooling API -----------------------

Rather than as done currently (returning a parsed model), it would be more desirable to have the hierarchy of ‘org.gradle.api.Project’ instances returned along with the path to the the settings.gradle and a reference to the ‘Project’ which is the project of the base directory (specified when loading the project).

If I have access to the ‘Project’ instances, it would be desirable to have some means to collect the dependencies of a project. The dependencies of each group (compile, runtime, etc.) should be fetchable separately and also their sources (and javadoc) need to be optional to fetch.

It would be nice if it would be possible to execute Gradle tasks by specifying, the objects of the previous paragraph. This would avoid the need to parse the build scripts again and executing a task could be more quick. Although in most cases, reparsing the projects is a more safe approach, there are cases where better performance is more desirable. Such a case is when executing the tests of a single class.

Accessing dynamically created methods and fields in Java is inconvenient, so a few convenience method for accessing these would be good. For example: A method which takes an object (e.g.: project) and the name of a property (e.g.: “sourceCompatibility”) and returns what where returned in the Gradle script (by calling: project.sourceCompatibility in the examples).

The main limitations of these is that Gradle must already be on the classpath and the user is tied to that particular implementation. One way to overcome this is to always return a WrappedGradleObject instance for objects of the Gradle API. This wrapper object can have methods to access fields dynamically. For example: Object getProperty(String propertyName). Objects whose classes are defined in the JDK should be returned directly (without wrapping).

Replacement

If you do not want to implement the above mentioned changes, the following things would still be necessary:

  • Access to the “sourceCompatibility” and “targetCompatibility”, as well as the

character encoding of the source files. - Being able to separately retrieve the dependency groups of a project (and also being

able to separately retrieve the sources). - Support for configurations of project dependencies. - Convenience methods to get all the dependencies (considering transitive dependencies)

of a project for a particular dependency group (e.g.: get every runtime dependency).

Cancellation ------------

Since executing a task can be very slow, cancellation would be desirable, so that accidentally started tasks can be canceled. If it is implemented, the cancellation should be based on something else than thread interrupts (e.g: cancellation tokens like in .NET).

2

Hey,

Thanks a lot for the feedback. It feels your requests might actually fit better our dev list.

  1. The reason Tooling API is specialized is because it needs maintain backwards and forward compatibility. The Gradle API moves forward with every release and tooling API prevents incompatibility problems.

  2. I assume you mean the performance when getting the project model with the dependencies when the dependencies have to be downloaded. Not sure what can be done here other than better progress reporting.

  3. Definitely we want to add ‘cancel’ feature to the tooling api operations.

  4. Lower level Tooling API. We think about the alternative tooling api which does not have forwards/backwards compatibility. It’s not on the table for the short time future, though. At the moment we’d rather the tooling api to model the use cases / workflows and keep the compatibility at bay.

  5. Regarding the sourceCompatibility/characterEncoding, dependency retrieval, project configurations requests - can you take a look at our jira? Some of the tickets have been already reported and certain discussions started. If something is not yet mentioned please create a separate forum entry for each. I’m keen on understanding the use cases and motivations behind the them.

3

I should be able to download dependencies in a separate step. That is, I first retrieve a model, then allow me to resolve the dependencies later group-by-group (compile, runtime, etc.). This would improve performance because the user probably does not need everything (e.g.: runtime is rather pointless for editing the code) immeditately but the compile time dependencies (also for non-project dependencies, the sources are not required right away). So after I have downloaded the dependencies, I could continue download other dependencies in the background and let the user work on the code. I myself would prefer a model which looks more close to the native model of a Gradle project rather than a specialized one (Idea and Eclipse).

There are quite a lot of reports about sourceCompatibility and tooling api, I cannot add much more to those. Since sourceCompatibility and targetCompatibility are amongst the most important attributes of a java project, I feel that IdeaProject having a constant (1.6) language level when not specifying explicitly (using the “idea” plugin) is rather surprising and makes this attribute pointless in the model. That is, requiring the user to specify something which was already specified in the script seems very wrong. Also from the user’s perspective, a requirement to use the “idea” plugin seems surprising in a NetBeans plugin.

About character encoding: It is a rather important attribute to parse the source code. Not everyone uses iso-8859-1. Although using a 1 byte/char characterset instead of another similar is “only” a visual problem. I assume that chinese and japanese people does not even use such character set. As far as I know, currently Gradle uses a platform dependent default character set for sources. Platform dependent character set is highly undesirable in case of an open source project, a better solution would be to use a constant (preferably “UTF-8”).

4

I have created example interfaces for what I would find a desirable model in the Tooling API: https://github.com/kelemen/gradle-model-example

If you find it useful for any purpose, you may use it. Note however that the source probably need to be adjusted (also I have only spent a few hours designing and documenting it, so there might be unwanted holes in the design).

5

I should be able to download dependencies in a separate step.

This request makes sense to me. I don’t think we will implement it in the short term future, though. The reason is the slowness is only once, when the gradle cache is yet empty. Also, in majority of cases, the compile dependencies are very similar to runtime. So if compile dependencies are already downloaded, getting dependencies for other configurations is not a stretch.

I acknowledge to feature request but I don’t think we will pick it up soon.

I feel that IdeaProject having a constant (1.6) language level when not specifying explicitly (using the “idea” plugin) is rather surprising and makes this attribute pointless in the model.

I agree completely. This is something we want to fix in short term (we have a story for that and want to play it in upcoming iterations).

About character encoding

I’ll discuss that with the team. How do you model this attribute at the moment in the build scripts? Via the compileJava.options.characterEncoding?

6

Yes, I set the encoding this way: “[compileJava, compileTestJava].options.encoding = ‘UTF-8’;”. I don’t know of other ways to do it in Gradle and everyone recommends this way when looking it up using Google.

7

That’s ok, just wanted to make sure how you configure it. I’ll get that back to the team, we might need some explicit modelling of the source encoding.