How Can I Automate the Detection Of Dependency Leakage?

We have a project with a ton of dependencies. We need to make sure the libraries we deliver don’t leak transitive dependencies to consumers. Some of the team members are experienced enough with dependency management to recognize when to correctly choose api over implementation. But the majority are not. To have our senior dev pore over everything manually would be overwhelming for them. It would also be a waste of a good resource. We need a way to go through our project and figure out what should be api and what should be implementation. To have our more inexperienced devs do that manually would be too error prone and too time consuming. So we need the process to be automated.

What are some options? Thank you.

Hey! What a small world :slight_smile: I’ve recently developed a Java CLI application that pretty much does what you are describing. You can find details about it here.

I call it the ABI Inspector. It automatically analyzes a library’s ABI to discover whether the dependencies declared in your project are exposed through the library’s public API.

Please, take a look at the details link above. That link lists the steps to run an automated inspection of your library’s classes and the modules they depend on.

If any api dependencies are found in the scan, the application will output a report that looks similar to Gradle’s and Maven’s dependency trees:

    ...
    
    >--: Project class com.lingocoder.poc.HttpClientWrapper has an ABI dependency on...
    |____\--- httpclient
    
    >--: Project class com.lingocoder.poc.Frankenstein has an ABI dependency on...
    |    +--- commons-math
    |    +--- de.huxhorn.sulky.generics
    |    +--- bitcoinj-core-0.15
    |____\--- janerics
    
    >--: Project class com.lingocoder.poc.Track has an ABI dependency on...
    |____\--- jackson-annotations
    
    >--: Project class com.lingocoder.poc.ProjectClass$1 has an ABI dependency on...
    |____\--- lingocoder.core
    
    ...

Here are the Maven repository details for either manual download (to run through the CLI) or to include in your build. Run a scan of your library project following the steps in the above details link. And let me know what if it’s what you had in mind?

I plan on turning the CLI application into Maven and Gradle plugins next. Feel free to PM me here with any feature suggestions, usage questions or feedback.


P.S. — Shout out to @trezkaj, whose StackOverflow question got me thinking about putting together the proof of concept for what’s now the ABI CLI application I link to above.

1 Like

Thank you. I had a look at that link. It looks like it could be useful for us. But some questions. In your example don’t some of the things get passed in two different times? Does the file you pass in the args array also need to have paths to the artifacts that are already defined in the dependencies? Or did I misunderstand the readme?

You did understand the usage example correctly. Good spot!

That duplication is partly due to a CLI application — that’s not a plugin — being retrofitted to work in a plugin context. And partly due to this being its very first release.

The way the CLI currently gets the info it needs to do its inspection, is not the ideal way; not yet anyway.

But I already have a better idea in mind about how to improve that for the second release. If you have any ideas of your own in mind, I welcome you to share them, by all means :slight_smile:

Were you able to have it scan your project successfully? I’d sincerely appreciate the feedback. Many thanks.

I ended up running one of our projects through the command line jar. It did not find any of our implementation dependencies

–: Project class com.github.xxx.finance.ctx.xxxConfig has an ABI dependency on…
|____— spring-context

–: Project class com.github.xxx.finance.sec.xxxToken has an ABI dependency on…
| ±-- webspec
| ±-- joda-time
|____— spring-context

–: Project class com.github.xxx.finance.mgt.xxxMgr has an ABI dependency on…
|____— spring-context

–: Project class com.github.xxx.finance.data.dao.xxxDAOImpl has an ABI dependency on…
|____— joda-time

–: Project class com.github.xxx.finance.data.util.xxxLoader has an ABI dependency on…
|____— spring-context

–: Project class com.github.xxx.finance.util.xxxFilter has an ABI dependency on…
| ±-- joda-time
|____— spring-context

–: Project class com.github.xxx.finance.data.xxxBuilder has an ABI dependency on…
| ±-- joda-time
|____— spring-context

It looks promising though. Thank you!

edit - the forum software didn’t render exactly what I pasted. The tree we got looks like your example.

Hey! Cool! Thanks for getting back. Yeah, the tool only tells you about the api dependencies it discovers in your project.

Basically any dependency not reported as an ABI dependency by the tool is, by implication, an implementation dependency.

There’s probably some C-S-ey mathematical notation that could express that more formally. But my memory on set theory notation is pretty rusty :slight_smile:

Would it be OK for me to PM you to get your feedback on why you ended up going the CLI route vs the plugin route? I’ll be making both interfaces easier to use. Any pointers you can give me would be appreciated. Thanks again.

No reason. I always choose the command line version of anything whenever I have a choise. I put 2 projects through it so far and I’m impressed how fast it finished! You might have something with this. When do you think an actual plugin will be available? Will it be able to do things like fail a build if I specifiy something as implementation that should be api? That would be a good feature to have for something like this.

1 Like

That is a fantastic suggestion! Thanks! I will think about how to implement that.

It’s difficult to say exactly when I will publish a dedicated ABI plugin. Possibly inside a month. But, if you (or any other lurking members of the community) can give me any pointers to what aspects of Gradle I should look at to be able to do that kind of thing, that would jump start my development sooner.

Before I start work on that though, I want to make sure the CLI is good-to-go first. Plus ideally I would prefer to have some more users test the CLI to get more super ideas like yours for features to include in a plugin version.

Speaking of which, I will be refactoring the CLI later this evening to remove some of that duplication you spotted earlier. It will no longer require that -d <dependencies-file> it currently does. I will probably be pushing that out as a new release sometime later tonight. Keep an eye out for v0.4.8

Thanks again! And please keep the feedback coming? :+1: