My project is a Verilog/SV language project. In Verilog, multiple versions of the same library at the same time in the same project are very common (the language and all the compilers are built to support that). How do I get gradle to download different versions of the same lib regardless of the version conflict (which is not a conflict in Verilog), and without flattening the dependency graph?
Gradle’s dependency resolution wants to resolve to a single version for a given configuration, so your best option may be to just split your duplicate dependencies like so:
repositories {
mavenCentral()
}
configurations {
compile5
compile6
}
dependencies {
compile5 'org.osgi:org.osgi.core:5.0.0'
compile6 'org.osgi:org.osgi.core:6.0.0'
}
task libs(type: Sync) {
from configurations.compile5
from configurations.compile6
into "$buildDir/libs"
}
Obviously, this gets more hairy as you have more and more “duplicate” dependencies and start to consider transitive dependencies. Do you have any examples of how this is normally handled with the current Verilog ecosystem? I assume it looks a lot like the other native languages.
I am jealous of all the amazing software tools that never quite work for Verilog coders! But this is an opportunity for Gradle maybe?
Answering your question, there no concept of automatic dependency fetching in Verilog. Designers manually acquire all the libraries they need and store them in a common location on the file system. Transitive dependencies are also acquired manually and needless to say, painful.
And you are right, this gets hairy when transitive dependencies are in the picture. And there are a lot of them, from vendors and suppliers as well as internal.
In Verilog, different versions of the same library are compiled to different locations on the file system and are given different names. The user writes a config/endconfig code block that specifies the mapping between library elements (verilog modules) and the library they come from. The Verilog “linker” reads this mapping as it instantiates the various verilog modules of the design in their rightful place.
I am hoping the Gradle dependency resolver can be modified or configured to fetch everything. In essence, I need a way to make it dumb! The existing Verilog tooling can handle the rest.
To sort of follow along with what is already happening, the dependencies could be published under different names. So if it’s expected to support org:depA:1.0 and org:depA:2.0 at the same time, you’d publish them as “org:depA-v1:1.0” and “org:depA-v2:2.0”. This is the pattern you see with Scala/Groovy a lot of the time. You’d be able to keep the single configuration and the transitive problem “goes away” if it’s OK to resolve to a single depA-v1 version and a single depA-v2 version. i.e., if you pulled in depA-v1:1.0 -> depB-v1:1.0 and depA-v2:2.0 -> depB-v1:1.5 and that was resolved to depA-v1:1.0, depA-v2:2.0 and depB-v1:1.5.
We’re still working on native support for Gradle and really baking in the idea of variant-aware dependency resolution. I’m not sure quite this idea has come up before. I think the unique names or some sort of unique variant dimension makes the most sense vs resolving multiple versions of the “same” dependency.
Is this a hobby/prototype project? Do you have any toy projects you’re trying this on that you’d be willing to share? (This piques my own personal interest.)
Thanks for your interest! I do not have a prototype project nor a hobby project, but at the same time I have no code either, I am just working how it would play out. I could create two versions of a project with just a readme file, upload them, and try to get gradle to download them both in a second project, but I know the answer.
I do not control other organizations’ library names and versions, and as much as I could re-wrap their libraries by “nameVersion” as you suggested, it is unlikely to be sustainable in the long run. Also, version differences in hardware also mean different power consumption and different physical space requirements, so when I need v1.1 in one place and v1.2 in another, it is really meant that way.
From another angle, multiple versions do co-exist in ~/.m2 and ~/.gradle, so these folders are okay with multiple versions. I am thinking that how it is managed after that is a concern for the language specific plugins, right? Or maybe it is engrained deeper in gradle?
I didn’t mean to rename the other libraries, I was just thinking of the packaging and how they would look in the artifact repository. e.g., you either have a build script that can compile the libraries and package them into something and upload that or you have a build script that takes an existing binary package and upload that to a Maven-style repo with some set of group:artifact:version that we could download later as a dependency. Since you’re having to manage that manually now (download/install to particular directories), that could be a small step forward if you had a single place and name for things.
When you say “version differences”, are you literally talking about a situation like v1.0.0 is the original release, v1.1.0 is the “low power” release, v1.2.0 is the “small packaging” release? Where using v1.2.0 when you need v1.1.0 is wrong because “small packaging” doesn’t necessarily mean it’s also “low power”? Or v1.2.0 isn’t a compatible replacement for v1.1.0 because something’s missing feature wise? I think that’s combining two different ideas–the revision of the library and a variant dimension of some kind.
If you’re saying it’s more like I have a dependency org:name:1.0.0 and it has variants like “low power”, “small packaging” and “default” and for a given project I need to select one of those, I think that’s right up our alley.
Even if it’s multiple variants per project, I think that’s OK (we’d need some way of selecting a particular variant to some smaller component of the project, but we’re doing something like that for C/C++ now).
The files in ~/.m2 and ~/.gradle are just caches. You could have different configurations or projects using different versions of an artifact at the same time. The idea of dependencies for a configuration resolve to a single version for a given group:name is pretty ingrained. I think it’ll be slightly different for the variant-aware dependency resolution, but I’m not sure how multi-versions fit in yet.
When you say “version differences”, are you literally talking about a situation like v1.0.0 is the original release, v1.1.0 is the “low power” release, v1.2.0 is the “small packaging” release?
That would be a possible but pathological case. Whoever created that library should have used a variant, not a version difference. But I never know what libraries I get until I see them.
Or v1.2.0 isn’t a compatible replacement for v1.1.0 because something’s missing feature wise?
This is the more common case, and they may be both instantiated in different places in the chip.
Multiple variants per project happen today. It very common with memories for example.
I would like to add that in a scenario where multiple versions of the same library are explicitely specified, version ranges would not be used, but snapshot versions might. This might help simplify the problem.
I think the direction we’re going in is that a project can have several components and each of those components can have separate dependencies. You can see that in the native support now and it’s emerging in the new JVM plug-ins. I could imagine a model where you have components “mem1”, “mem2”, “alu”, etc. And then you have your “chip” component that depends on the other smaller pieces with tooling that could either compile each smaller component separately or aggregate them up into a single compilation (generating Verilog’s config/endconfig input?). It sounds hard and fun.
That’s equivalent to my first response. group1/group2 are two different configurations in that Ivy example (roughly the same as Gradle’s configurations)