Multi Repo issue with API vs Implementation (Gradle 5.x)

Hey looking for some help wrt multi libraries, repos and apps

Here is the scenario, trying to upgrade our build system to use Gradle 5.x etc

we have multiple libraries (e.g. common git repo). Also there are multiple apps (android apps, each one in it’s own repo). We typically build all common libraries and those are placed in Local Maven Repository. These libraries are consumed by APPs. There are library dependencies etc. e.g. assume that LibrayA --> LibraryB (–> uses or depends on) and LibraryA --> LibraryC, … . In current Gradle 4.x setup we use ‘compile’ and include ‘LibraryA’ in one of our APPs and I can use all the classes from LibraryA, LibraryB …

Now I changed it to use ‘api’ in LibraryA’s build.gradle e.g. api project (’:LibraryB’), …

But for some reason when I build my Apps which included LibraryA (implementation LibraryA), they are not able to see other dependent Libraries. They are built and available local maven repo. My app builds if I include LibraryB, … in the App build.gradle. That’s pain and I am not sure why it does not work.

Also my question is when I define ‘api’ what happens to transitive dependencies. They are packaged with my library, in the above example LibraryA, or Gradle tries to look for my LibraryB, LibraryC etc in localMaven repo or external maven repos? How does this work? How to debug (when I checked gradle dependencies for my app project, LibraryA is shown but not expanded to show it’s dependencies). What could I be doing wrong here?

All the above works if I have one project e.g. App, and LibraryA, LibraryB etc are also part of the same project. But in my case my library project is different from my App project, glue is my LocalMavenRepo.

Thank you, any help is appreciated.

Sai

What you say here…

…would suggest, therefore, that all your common libraries have Maven-like coordinates such as com.cloversai:LibraryX:n.n. Then semantically they aren’t really projects in the project(:LibraryX) sense in which you’re trying to reference them. Semantically, they are what Gradle calls „modules“.

In that case, this…

…would not be the correct way to specify module LibraryA's dependency on module LibraryB

I would try something like this in the build.gradle of LibraryA instead:

    ...
    repositories{
        mavenLocal() // caveat Gradle advises against using mavenLocal()
        ...
    }
    ...
    dependencies{
        ...
        api com.cloversai:LibraryB:1.0
        api com.cloversai:LibraryC:1.0
        ...
    }
    ...

See Gradle’s docs on Dependency Resolution. In a nutshell, Gradle fetches them from one of the repositories{} you specify and then persists it to a local file system cache. Then when Gradle builds the library it adds the respective dependency’s location in the file system cache to the library being built’s classpath.

A library’s dependencies are not packaged with the library.

I can’t explain it better than the Gradle documentation already does. But my working understanding of it is that compile, api and implementation are what Gradle calls Configurations. The compile configuration is deprecated. Gradle recommends that it no longer be used because it leaks dependencies. That is a bad thing. It bloats your classpath and that bloat has a negative impact on your build; among other things.

The api and implementation configurations were introduced to be used instead of compile. Using them appropriately has a positive impact on your build; among other things. See the docs for details.

So, if I understand correctly what you described, your build.gradle for your app should look something like:

    ...
    dependencies{
        ...
        implementation com.cloversai:LibraryA:1.0
        ...
    }
    ...

But not being able to see the actual build.gradle of your app and libraries, the above is just my best guess.

Thank you very much, will try your suggestion api com.cloversai:LibraryB:1.0. Actually all our libraries are published to maven local as SNAPSHOT versions, they following naming convention like Maven-like coordinates. I thought api project(':LibraryB') makes gradle to build LibraryB first and then build LibraryA. Not sure how api com.cloversai:LibraryB:1.0 works? Doesn’t this expect LibraryB of version 1.0 be already built and present in MavenLocal?

You might want to look into Gradle’s Composite Builds feature.

According to the examples in that documentation, you should be able to do something like…

    cd /path/to/LibraryA
		
    gradle --include-build /path/to/LibraryB :someTask

If that doesn’t work for you, please get back and share whatever solution you do find that does what you need?

lingocoder,

Finally I was able to create a dummy project that mimics our project structure and the problem that I am facing can be replicated using it. I am using gradle 5.4.1 for my experiments. (wanted to upload my projects, its not letting me as I am a new user). But here is my google drive link sample app zip file. These projects created using Android Studio.

So if you look at this structure, there are two main directories, one that contains an app and bunch of libraries. Another with a lone app. Now when you build ‘LibrariesProject’, it should put all the libraries into LocalMaven repo (~.m2/repository/com/experiments/librarya …).

If you look at ‘AnotherApp’, which uses LibraryA, it can instantiate MyClassA, but it can’t see MyClassC (look into MainActivity.java). So something is wrong with my structure or build.gradles etc. Any help is greatly appreciated.

Thank you

Thank you for sharing the projects, @cloversai. It’s always very helpful to see build.gradle scripts like this.

To the best of my knowledge, you have two options:

  1. Don’t change anything. Except simply uncomment implementation 'com.experiments:libraryc:1.0.0' and you’re done.
  2. In your library projects, implement the project-to-module semantics change I described in the first part of my first reply — api project(LibraryX) to api com.cloversai:LibraryX:n.n). That would require you to also change to using the Composite Builds feature I linked to yesterday.

There might be other options available that I personally don’t have any knowledge of or experience with at this point. So I invite any other more knowledgeable forum members to feel free to chime in.

When you do get things working the way you want, @cloversai, please share it with everybody here so we also learn from your experience.

Thanks, but option ‘1’ you are suggesting does not work for me. I have to list so many dependent libraries, in so many apps that we maintain, that’s real pain.

Hope someone from Gradle development community answers my questions???

I understand you reason for nixing option ‘1’. Now, please, help me understand your reason for ignoring option ‘2’?

Your answer could speed up the Gradle development community steering you toward a better, more informed, solution sooner.

Will explore Option ‘2’ and let you know.

Looks like just an ‘api’ is not a solution here for a multi project use case.

One more thing I didn’t like is it compiles, as long as I don’t use MyClassC in my ‘AnotherApp’, app crashes during run time when I use ‘MyClassA’. This is real bad, as developers have no clue LibraryA depends on LibraryC internally, probably Gradle should address this issue (in the case of same project, it works perfectly fine, it somehow includes LibraryC and builds the app, in my example App inside my LibrariesProject just works as expected just by including LibraryA)

My sense is that a large part of the problems you’re encountering, are to do with your confusing a project as being a library.

I’ve tried to explain why you shouldn’t conflate the two. They’re different, semantically. One is a „binary dependency“. The other is essentially a file system directory dependency.

Personally, I wouldn’t expect to be able to interface with two fundamentally different types of dependencies in the same way. The problems you describe actually bear out my expectations.

Since you have a project that reproduces the issue, you could raise a ticket with Gradle. Who knows? You might have discovered a bug. If nothing else, you might at least get expert guidance on what the correct solution is for your requirements.

On top of that, I can’t recommend often enough that you read and thoroughly understand the Gradle documentation. Specifically: Configurations, api/implementation, Composite Builds. The search box on the left-hand side of the documentation can be a life saver.

Hey @lingocoder , thanks a lot for your guidance, I really appreciate it.

It’s all started with simple assumption (probably wrong one) replace ‘compile’ with ‘api’ it works while upgrading to Gradle 5.x. As ‘compile’ was working with our directory structure, mavenLocal() etc with Gradle 4.x. Now I am learning just replacing ‘compile’ with ‘api’ does not do the same thing for my use case.

Sure will go through the documentation, thanks again for your help.

What were your original reasons for wanting to replace ‘compile’ with ‘api’ in the first place?

Hey I might have found the real issue, the POM file generated does not have dependency information. I am using maven-publish, probably doesn’t have right settings. May be that’s the reason ‘api’ not able to figure it out. I will try that and report.

That’s it, it works @lingocoder, I manually added LibraryC dependency in the .pom file that was generated for my LibraryA, it compiles now, wooh, it was mistake on my end (which I couldn’t figure out earlier), now will figure out how to generate dependencies in the .pom file.

thank you this discussion helped me.

Good luck with that :slight_smile:

Also: What were your original reasons for wanting to replace ‘compile’ with ‘api’ in the first place?

because ‘compile’ was deprecated and gradle has been telling us ‘compile’ will go away with 5.x and replace it with ‘api’ or ‘implementation’. Looks like they are keeping ‘compile’ still probably will go away soon.

So @cloversai? Are you ready to share your breakthrough of how you got Gradle to write out your library’s dependencies into the maven-publish plugin-generated *.pom file?

You sharing your solution will help all of us in the community here when we eventually need to do something similar. Thanks in advance for sharing.

All I had to do was start using this plug in gradle-maven-plugin. Or one can modify their build.gradle files to generate additional dependency information. Look at pom-manager for inspiration on how to do this.

Thanks for sharing that :+1:

I have a couple requests for you:

  1. Can you help me understand what it is about the Composite Builds option you don’t like? Please?
  2. Can you share to Google Drive an updated version of your sample app that implements your plugin solution? Please?

Or maybe I should word Q1 another way: What is it about that plugin that you like more than you like Composite Builds?

Your answers might be taken into consideration by Gradle’s Composite Build devs (@st_oehme?/@daz?/@sterling?) as ideas for future improvements.

TIA.

See if you have all libraries in the same project and specify project dependencies or if you use just Java Library projects Gradle takes care of transitive dependencies and also generates dependency information in the .pom file. But not sure who’s responsibility it is if it’s an Android Library. That part is not clear, as a Gradle user, I would expect behavior to be same or be explicit. As I wrote earlier, missing Library information is evident during Run Time not during compile time (esp with Android Dependency information). The code that I shared already shows this, but that does not have fix. Fix is just including that plugin.