How to include a local jar file as dependency

Help/Discuss
1

Hi, I’m trying to include a local jar file as a dependency, and nothing is working. First, here is the build.gradle.kt for the library:

plugins {
    id("org.jetbrains.kotlin.jvm").version("1.9.20")
    `java-library`
}

version = "0.1.0"

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.apache.commons:commons-math3:3.6.1")

    testImplementation("org.jetbrains.kotlin:kotlin-test-junit5")
    testImplementation("org.junit.jupiter:junit-jupiter-engine:5.10.0")
    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}

java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(21)
    }

    withSourcesJar()
    withJavadocJar()
}

tasks.named<Test>("test") {
    useJUnitPlatform()
}

It exposes a single method named exampleLibraryMethod(x: Int): Int which acts as a wrapper for an absolute value function in commons-math3. I ran the build task and copied the 3 jar files in build/libs into a libs folder in a secondary project.

This secondary project has the following build.gradle.kt:

plugins {
    id("org.jetbrains.kotlin.jvm").version("1.9.20")
    application
}

repositories {
    mavenCentral()
}

dependencies {
    implementation(fileTree(mapOf("dir" to "libs", "include" to listOf("*.jar"))))

    testImplementation("org.jetbrains.kotlin:kotlin-test-junit5")
    testImplementation("org.junit.jupiter:junit-jupiter-engine:5.10.0")
    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}

java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(21)
    }
}

application {
    mainClass = "org.example.AppKt"
}

tasks.named<Test>("test") {
    useJUnitPlatform()
}

With this setup, I’m getting an “unresolved reference” error implying that the library is not actually being included, or it was built incorrectly. I’ve tried:

  • Using flatDir { dirs("libs") }
  • Using files("libs/mylib.jar")

Is this an issue with how I’m building the library, or with how I’m including it?

2

Hard to say where the issue is, with only seeing the build script.
Did you check the class you expect is actually present in the jar?
Could as well be, that you put the source files to the wrong directory and thus they were ignored.

Besides that, if you really have to depend on a local file dependency, you should neither use fileTree nor files. While they will work, they have several drawbacks and shouldn’t be used, but a flatDir repository instead. The flatDir repository alone does not yet add any dependencies, it just adds the repository, you still need to add a dependency with that, as documented for flatDir.

But actually avoiding local file dependencies would be best, for example if you do it like that, you miss all dependency information you have for your library jar, so you will for examle miss the commons-math3 dependency at runtime unless you also depend on that even if your custom class is found.

You should probably have a look at composite builds which micht maybe fit your use-case.

3

The library does indeed contain the class:

$ jar tf lib-0.1.0.jar
META-INF/
META-INF/MANIFEST.MF
org/
org/example/
org/example/LibraryKt.class
META-INF/lib.kotlin_module

How do you know that fileTree or files have drawbacks? Do you have any sources? Most documentation online suggests to use these methods, so I don’t understand what you mean by drawbacks.

I tried your method, per this documentation, and it didn’t work, same result:

repositories {
    mavenCentral()
    flatDir {
        dirs("libs")
    }
}

dependencies {
    implementation(":lib:0.1.0")
    ...
}

I cannot avoid local dependencies. I have thousands of lines of code that I reuse across dozens of projects (and more on the way). I’m not about to copy that code to every single project, I’m pretty sure Java libraries exist specifically to avoid having to do that. I feel like I’m not asking for anything complicated here, just to create a Java library and then use it locally in another project. This particular example is contrived to figure out how to achieve that goal.

And I am hoping to run a Nexus container on a server to expose dependencies as artifacts in the future, however I should be able to at least do it this way first.

I’m prepared for issues with transitive dependencies, such as commons-math3 not being present, it just doesn’t feel like I’ve reached that point yet. For example, I’m not seeing my local library under “External libraries” in the Intelli-J Project view. Additionally, adding that math3 dependency to my consumer project does not fix the issue.

If the build scripts are not enough, I’ve uploaded the project to the following link: (removed, see edit #2 below). I can’t upload it here as I’m a new user. If you’d like me to share other parts of the code so as to not click a dubious link, I can do that as well.

EDIT: I renamed the library to mylib.jar at one point during this, but I changed my paths accordingly so that shouldn’t have been the issue. My shared project has the names reverted to lib-0.1.0.jar.

EDIT 2: After this comment I gained the needed trust level to upload this here:

example-kotlin-project.zip (234.6 KB)

4

How do you know that fileTree or files have drawbacks?

Well, I’m a seasoned Gradle expert using it for many years already since its pre-1.0 phase.
So I simply have enough experience to know they have significant drawbacks.
You for example miss them in build scans.
And they are also not shown in reporting tasks like dependencies and dependencyInsight.
Just don’t use them, but prefer a flatDir repository if you really really need local file dependencies.

Most documentation online suggests to use these methods, so I don’t understand what you mean by drawbacks.

By “documentation” you probably mean random blogs, StackOverflow answers, and similar.
Those are often providing bad advice, or outdated information.
Also something that might have been idiomatic in the past could be discouraged bad practice today.
…
You probably came here to get some expert’s advice, so you got mine. :wink:

I tried your method, per this documentation, and it didn’t work, same result:

Yeah, it would have been strange if the result would be different than with for example files("libs/mylib.jar"). But at this point the shared information so far is not enough for me to make any educated guess. But as you luckily provided an MCVE now, the problem is clear. You try to find the lib in example-kotlin-app/app/libs while you put it to example-kotlin-app/libs. You can also see that from the error message clearly, which says it cannot resolve the dependency and the path it looked for, another advantage. After fixing "libs" to "../libs" or moving the files to the expected place, it compiles just fine.

I cannot avoid local dependencies.

Yes, you can.

I have thousands of lines of code that I reuse across dozens of projects (and more on the way). I’m not about to copy that code to every single project,

I never suggested this.
I just said that it is a pretty bad and cumbersome idea to manually build jars and copying them around,
suggesting a much better alternative that is exactly designed for that use-case.

I’m pretty sure Java libraries exist specifically to avoid having to do that. I feel like I’m not asking for anything complicated here, just to create a Java library and then use it locally in another project. This particular example is contrived to figure out how to achieve that goal.

Yes, you are not asking for something complicated and it works just fine if you don’t confuse paths, it is just a bad idea to do it like that.

You could for example instead publish to a local directory-based Maven repository (but better not mavenLocal() as that has a whole lot of other problems) and then use that repository in your other builds.

Or as I already suggested above, you could use composite builds if it is mainly to develop the lib and its usage in parallel as then Gradle just builds the lib before building the app fully automatically if necessary without the need to build jars and copy them around and without loosing dependency information, and you can also edit the code of both in the same IDE right away.

5

After fixing "libs" to "../libs" or moving the files to the expected place, it compiles just fine.

Thank you, this was my issue. My build is working now.

Yes, you can.

Can you elaborate on how I can avoid local dependencies? The options I see currently are:

  1. Compile library into local dependency jar for distribution. My hope was that I could publish that to the repository, and then Jenkins could pull it when building (and I could just download it every time I start a new project for local development).
  2. Clone project, build locally, and publish to maven local and use the resulting artifact, like you mentioned.
  3. Publish to a maven repository hosted on a server somewhere, using something like Nexus repository manager, Artifactory, etc.
  4. Use something like jitpack.io

In my work, I’m currently doing option 2, which you mentioned is bad. I assume it’d be “better” to create a specific maven repository locally, other than the default one? You also say option 1 is bad. I started going down the option 3 path, but started to wonder if I was over-engineering things at some point for what I’m trying to do (which really only affects me and potentially 1 other developer). My work does have a server which could host a maven repository, likely with nexus running in a Docker container.

It feels like you are suggesting a 5th option which is to utilize composite builds. I have a basic understanding of composite builds. For example, this folder structure:

~/Projects/
	my-lib/
		...
		settings.gradle.kt
	app-1/
		...
		settings.gradle.kt
	app-2/
		...
		settings.gradle.kt

I would include my-lib in app-1 and app-2 with includeBuild("../my-lib"). Is that what you are suggesting? Or are you suggesting to have an instance of my-lib in each consumer app, like this:

~/Projects/
	app-1/
		my-lib/
			...
			settings.gradle.kt
		...
		settings.gradle.kt
	app-2/
		my-lib/
			...
			settings.gradle.kt
		...
		settings.gradle.kt

The latter would more closely model the example on the docs, however the library is a Git repository. So, since the parent app is also a git repository, that would involve using git submodules correct? It’s fine if that’s the case, just want to make sure I understand correctly.

Or are you suggesting to develop apps in parallel under a separate “composite build” project, cloning both the library and app into the composite build project folder next to each other?

6

Can you elaborate on how I can avoid local dependencies?

I’m sorry if I was unclear.
I meant what you can avoid is copying jars around, using file-based dependencies and losing all dependency information.

Compile library into local dependency jar for distribution. My hope was that I could publish that to the repository, and then Jenkins could pull it when building (and I could just download it every time I start a new project for local development).

Sure, you can do that. But that is the same as option 3, isn’t it?
If not, then I probably misunderstood.

Clone project, build locally, and publish to maven local and use the resulting artifact, like you mentioned.

That’s not what I said.
I said a local directory-based maven repository that is not mavenLocal().
mavenLocal() has some serious problems and is broken by design in Maven already.
You can find further details about that at The case for mavenLocal().

Publish to a maven repository hosted on a server somewhere, using something like Nexus repository manager, Artifactory, etc.

That’s probably the most senseful and easy approach, you just need some server or VM somewhere and can run the free Nexus server there if it is only for internal usage.

Use something like jitpack.io

Be very careful.
If you really mean “something like” it might be ok.
If you mean JitPack itself, better don’t use it.
It is great for quickly trying something on some branch or commit that is not released somewhere.
But it is absolutely unsuitable as main publishing platform.
It is buggy, it is unreliable, it is slow, it can easily lead to confusion, it modfies build results in ways the often break the metadata, it is broken by design when used as main publishing tool as for example version ordering does not work out if you use commit ids as version, …

It feels like you are suggesting a 5th option which is to utilize composite builds.

Yes. :slight_smile:

I would include my-lib in app-1 and app-2 with includeBuild("../my-lib") . Is that what you are suggesting?

For example, yes.

Or are you suggesting to have an instance of my-lib in each consumer app

Also possible, however you need it.

So, since the parent app is also a git repository, that would involve using git submodules correct?

That’s one possibility, yes.
If app-1 and app-2 need differen versions of my-lib this might even be preferable.
Or you can also have two workspaces of my-lib with the different versions and include those in the app if you don’t like Git submodules.
Many ways, depending on concrete needs.

The point is, that with composite build you still do the “build locally the jar and use it in the other project” but without manually copying jars, without manually triggering that other build, and without losing dependency information. Just build the app and if necessary, the lib is built automatically first.

Or are you suggesting to develop apps in parallel under a separate “composite build” project, cloning both the library and app into the composite build project folder next to each other?

That’s not what I suggested, but surely also one possibility maybe.

7

You could for example instead publish to a local directory-based Maven repository (but better not mavenLocal() as that has a whole lot of other problems) and then use that repository in your other builds.

Ok you caught my attention enough to create an account, what is the problem with mavenLocal()? i’m using it and your claim got me wondering.

8

Maven Local is broken my design in Maven already. In Maven it is a mixture of download cache and local repository. So if you for example ever run any Maven build, it has the things that build downloaded available, but maybe not other artifacts like other variants or Gradle Module Metadata and so on. But Gradle if it finds the POM expects to see a complete repo entry, not a partial download cache. This can lead to problems or also cause build suddenly failing when before proper artifacts from a repository were used and suddenly there is something in Maven Local. Also having Maven Local in you repository list makes you builds slower. If you ever need to use Maven Local (if possible you should prefer composite builds or at least a dedicated local repository) you should always have it as last entry in the list and with a content filter, so that only exactly those artifacts you know are complete and expected to be taken are used from there. You can also read more details at Declaring repositories.

1 Like
9

Nice to know, Thanks!

1 Like