"Refresh Gradle Project" takes a long time

Help/Discuss Buildship
1

Hi,

I hava an issue which is more an interrogation on performances than a bug. Let me explain :

I created an ad_hoc project for a composite build and then I import it into Eclipse to use it with Buildship. The structure is :

  • ad_hoc
    • includeBuild ‘app’
    • includeBuild ‘frameworks’ (multiproject build ~70 projects)
    • includeBuild ‘libs1’ (multiproject build ~5 projects)
    • includeBuild ‘libs2’ (multiproject build ~10 projects)
    • includeBuild ‘libs3’ (multiproject build ~20 projects)
    • includeBuild ‘libs4’ (multiproject build ~15 projects)
    • includeBuild ‘libs5’ (multiproject build ~5 projects)
    • includeBuild ‘libs6’ (multiproject build ~3 projects)

“app” depends on sub-projects of “frameworks” and “libs"
Sub-projects of “frameworks” depends on sub-projects of “libs” and even sub-projects of “libs” depends on sub-projects of 'libs”

So it is a very big plate of spaghettis :slight_smile:

Eventually everything works as expected, Buildship does a very good job on dependency substitution. All my projects get linked to each other.

Except that when I hit “Refresh Gradle Project”, it takes for ever, especially dependy resolution which seems to be done from scratch everytime (and on every project ?). When I just launch some gradle commands on ad_hoc project with command line, it is quite faster (10 seconds compared to 3 minutes at least)

Did I miss something ? Also what could explain the differences between Buildship and CommandLine ?

Also I use Gradle 4.3.1

Thanks a lot

Julien

2

It shouldn’t take that long. Is there any chance you can reproduce the slowdown with some dummy projects that you could share? Otherwise you can attach one of the common profilers (JFR, JProfiler, YourKit) to the daemon (or to Eclipse, depending on where the time is spent) and send us a performance snapshot for analysis.

3

Hello,

Ok I tried to profile Eclipse before with JProfiler with no significant results. This time I profiled the daemon and it’s better.

I give you those 2 hotspots screen captures.

By the way, I didn’t mention I use the spring-dependency-management plugin with two in-house boms.

I’m doing something wrong for sure but I don’t know what.

HotSpotClasses
HotSpotClasses.PNG1920×1040 150 KB
HotSpotMethods
HotSpotMethods.PNG1920×1040 156 KB

The times are worse because of the profiling.

Thx

Julien

4

Make sure you use sampling (every 5ms should be fine) instead of tracing (which is only useful when looking at very few methods).

Can you upload a sampling profile so I can take a direct look at it? Only seeing a screenshot makes it hard to draw conclusions.

5

Yes here is the profiling sample

https://drive.google.com/open?id=19GsLLSsM8w2aCxmn-g6Zw4mb3GGhdYCa

Julien

6

This was taking using instrumentation, which is not useful due to the high overhead. Please create another profile using method sampling.

7

Allright it’s very different

https://drive.google.com/open?id=17oqaY8PFmwSHnQkQUOKOoukwWBTTU0lf

Thx for your time

Julien

8

Can you give Gradle 4.5 RC1 a try? I just optimized the code path that I see taking a lot of the time in the snapshot.

9

Well the first sampling was on windows which could explain de bad file performances

Here is another on linux. The performance is better but it’s still around 1 minute. I give it to you (it’s different again)

https://drive.google.com/open?id=1NM8H8gbIWy5ovt7URyuU6vTb7RTcLBHB

I’ll try the new gradle (I think we have issues with custom classifiers which mess up with composite buildsin gradle 4.4 , maybe it’s fixed)

Thx

Julien

10

On your Windows machine, do you have a virus scanner that might be watching your temp directory? Because as far as I can see only the access to your temp dir is slow.

The overhead in general should be lower with Gradle 4.5.

Beyond that most of the time is spent in the Spring dependency management plugin. In Gradle 4.5 we actually added experimental BOM support to Gradle itself, so you could try replacing the Spring plugin with that feature. You can try it out like this:

  • Add gradle.experimentalFeatures.enable() to your settings.gradle
  • Instead of using dependencyManagement, just add the BOM as a normal dependency, e.g. implementation 'com.mycompany:mybom:1.2'
11

Hello,

I was supecting the virus scanner but I didn’t think of the temp directory. It really improves performance (but it is still good)

I tried boms in gradle 4.5, it’s “working” (I can’t launch my project in Eclipse but it’s not gradle fault I think; it’s a very legacy project and even minor changes can break it). The boms management does not seem to support “properties” like for version parametrization as in spring plugin. Do you plan to support it ?

Thx

Julien

12

There is no plan to support properties in BOMs at this point. It would be interesting to see when that is useful. At first glance it seems to water down the contract defined by the BOM.

Feel free to attach a snapshot using Gradle 4.5 (with the spring plugin or with the experimental BOM support). I’ll take another look then.