How do we get the IDEA project to contain Maven library references, not direct references to jars?


(Trejkaz (pen name)) #1

In our project, the .iml files and .idea/libraries/*.xml files are currently being generated from the .gradle files.

What we find is that the library files all reference the .jar file directly in the cache. Then, because they are doing this, after any particular update from source control, we can find the project is suddenly in an unusable state. Since IDEA itself has numerous bugs which appear to make it impossible to refresh from the Gradle project, this means one more command to run manually from the command-line any time anyone might have made a change, which is to say, every single update from source control.

What I would prefer to see is the libraries configured as proper maven libs in the IDEA project. This way, after updating, IDEA would hopefully realise which libraries it hasn’t fetched the jars for yet and download them automatically. Then hopefully it will be OK to expect things to just work, which is pretty far from what we have right now.


(Benjamin Muschko) #2

Since IDEA itself has numerous bugs which appear to make it impossible
to refresh from the Gradle project, this means one more command to run
manually from the command-line any time anyone might have made a change,
which is to say, every single update from source control.

You would only have to rebuild the project files if any of your dependencies has changed. Is that the case? If you think there’s a bug in the IntelliJ, it would make sense to report that back to Jetbrains so they can fix it.

What I would prefer to see is the libraries configured as proper maven
libs in the IDEA project. This way, after updating, IDEA would hopefully
realise which libraries it hasn’t fetched the jars for yet and download
them automatically.

IDEA does not support this.


(Trejkaz (pen name)) #3

Yes. This happened 6 times yesterday out of the 9 times I updated. So I “only” had to rebuild the project files 66.66% of the time.

When I add a library in IDEA, I get a different XML file to what Gradle creates and IDEA shows a little Maven icon in the library screen next to the libraries created properly in IDEA. If you look in the XML itself, files created by Gradle reference $HOME_DIR$ instead of $MAVEN_REPOSITORY$ and Gradle also “forgets” to insert this:

<properties maven-id="net.java.dev.jna:jna:4.2.0" />

So I don’t think it’s IDEA that does not support this. Rather, perhaps it’s Gradle that doesn’t support this?


(Benjamin Muschko) #4

When you say “add” does this mean you add the dependency manually? I doubt the Gradle integration in IntelliJ understands that. My guess what happens here is that IntelliJ thinks you are working in a Maven project and resolves just this dependency with Maven instead of Gradle.

What I’d propose to do instead is to generate the project files with the idea plugin once. Then you import the project into IntelliJ and activate the Gradle support in IntelliJ (make sure you select the file-based project format). From there on you can add new dependencies to your build script and press the “refresh dependencies” button in the Gradle integration tab. That should automatically sync your project files. This workflow also works if you updates from SCM.


(Trejkaz (pen name)) #5

So is there any particular reason why Gradle can’t generate the library XML files with the proper maven metadata in them? Or do you just think that the way you do things in your team is the way everyone else should have to do them in theirs?

In our case, we wanted to avoid multiple users having to manually hit refresh every time one committer happens to update a library. Because it isn’t always clear when a library has been added, it quickly became the case that every time you do an SVN update you were forced to hit Refresh just in case someone did change it, which is a manual step we would rather not have to perform.

Because of this, we figured the one person who adds a library should be responsible for updating all the .iml files as well, so that nobody else would have to. And it mostly works fine, aside from Gradle not adding that one tag which IDEA uses to know where the library should be downloaded from. :frowning:

(And in terms of clearing up the misunderstanding over “When I add a library”, I was referring to when I use IDEA in the absence of Gradle, as a demonstration that when you avoid using Gradle entirely, IDEA is perfectly capable of syncing downloads from a Maven repo.)


(Benjamin Muschko) #6

So is there any particular reason why Gradle can’t generate the library XML files with the proper maven metadata in them? Or do you just think that the way you do things in your team is the way everyone else should have to do them in theirs?

The idea plugin doesn’t do that at the moment. I think if we would add that, the Gradle integration in IDEA wouldn’t be able to evaluate it or use Maven to resolve it. We want to avoid these situations. If the Gradle integration in IDEA would introduce a syntax that it could use then it wouldn’t be a problem and we could think about using that instead. At the moment it’s just not clear to me what IDEA does with the property syntax exactly. You might want to ask this on the Jetbrains forum.

In our case, we wanted to avoid multiple users having to manually hit refresh every time one committer happens to update a library. Because it isn’t always clear when a library has been added, it quickly became the case that every time you do an SVN update you were forced to hit Refresh just in case someone did change it, which is a manual step we would rather not have to perform.

That’s a step the IDE should provide for you. The Maven integration in IDEA already does that anyway. You edit the list of dependencies, IDEA notices the change, asks Maven to download the dependency and adds it to the project metadata. The Gradle IDEA plugin simply doesn’t do this at the moment and require the manual step.

Because of this, we figured the one person who adds a library should be responsible for updating all the .iml files as well, so that nobody else would have to. And it mostly works fine, aside from Gradle not adding that one tag which IDEA uses to know where the library should be downloaded from.

Our perspective on this is that the build should be the source of truth. Information used by the IDE should be derived from there. If someone adds a library manually to the .iml file, the build wouldn’t know anything about it and therefore fail to build properly on the command line.


(Trejkaz (pen name)) #7

Our perspective on this is that the build should be the source of truth. Information used by the IDE should be derived from there. If someone adds a library manually to the .iml file, the build wouldn’t know anything about it and therefore fail to build properly on the command line.

We’re taking this approach as well, we just happen to also believe that a given unit of work should not be performed more than once.


(Benjamin Muschko) #8

We’re taking this approach as well, we just happen to also believe that a given unit of work should not be performed more than once.

I totally agree. Having a reload mechanism for IDEs that automatically checks for changes in the build and applies them is a must. I am not quite sure if that functionality is exposed through the Tooling API yet but once it is IDE providers would be able to make use of it.


(Trejkaz (pen name)) #9

This stuff continues to be a problem months down the track.

Every time someone updates something, it doesn’t get downloaded unless you have run the Gradle build at least once. Because you don’t know whether the update changed any build files, you ultimately are forced to do a rebuild every time you update to be sure you have all the jar files in the cache.

This build then takes about 30 minutes. Part of that is just that Gradle is slower than Ant. Part of it is that it has to download jar files that Ant didn’t. Part of it is that it seems to take 10-20 times longer to download javadoc jars than the binary jars. But in combination it takes about half an hour to regenerate the IDEA files sometimes. (Unless you know exactly what module the change occurred in and can run just that one, but usually you don’t.)

Half the time, after doing this, IDEA doesn’t see the jar file even though it’s there, which is some separate IDEA caching bug going on. Even invalidating caches doesn’t always help and one developer here even had the issue mysteriously stop happening at 5pm one day after he had spent literally the entire day trying to get IDEA to work.

So overall this hampers our ability to pull changes from upstream, which used to be trivial when we had the dependencies checked in. Now we find ourselves deliberately avoiding updating from upstream because if we do, Gradle might waste from half an hour to a day of our time.

If only it had put the proper metadata into the .iml files so that IDEA could just download the files. If only IDEA’s Gradle support didn’t result in broken project files. If either of these things weren’t completely broken, it might be workable, but it isn’t.