Limitations of Gradle source dependencies


(Peter Palaga) #1

Hi, I am the author of srcdeps, a source dependencies implementation primarily for Maven [1] but I also created rather a hacky PoC for Gradle [2]. Having seen your announcement [3] I am experimenting with your implementations and try to figure our where its limitations are. Here are my questions:

(1) There is no way to depend on a specific commit sha1, right? Are there any plans to add this?

(2) Is there a way to depend on a Maven/Ant/sbt project?

(3) Knowing that source dependencies use composite builds under the hood, I am trying to figure out if the use of the artifacts produced by source dependencies is somehow limited. E.g. if I can include the dependency jar into a war. Or if I can get the jar and unzip it. I tried something like the following:

// settings.gradle
rootProject.name = 'gradle-source-dependencies-demo'
sourceControl {
    gitRepository("file:///home/ppalaga/orgs/srcdeps/demos/181011-hello-gradle/.git") {
        producesModule("org.srcdeps.hello:hello-gradle")
    }
}
// build.gradle
repositories {
    mavenLocal()
    mavenCentral()
}

apply plugin: 'java'
apply plugin: 'maven'

dependencies {
    implementation('org.srcdeps.hello:hello-gradle') {
        version {
            branch = 'master'
        }
    }
    testCompile 'junit:junit:4.12'
}

sourceCompatibility = 1.8
group = 'org.srcdeps.gradle.plugin.quickstarts.srcdeps-gradle-dep-gradle-git-revision-quickstart'
version = '0.0.1-SNAPSHOT'

def unzippedDir = "$buildDir/dependencies/unzippedDir"

task unzip(type: Copy) {
    configurations.implementation.asFileTree.each {
        from(zipTree(it))
    }
    into unzippedDir
}

build.dependsOn unzip

But I got this error:

...
Caused by: java.lang.IllegalStateException: Resolving configuration 'implementation' directly is not allowed
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.assertResolvingAllowed(DefaultConfiguration.java:1060)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.access$1500(DefaultConfiguration.java:123)

Is there a way to get the jar somehow? Or is this not possible due to how composite build works?

Thanks,

Peter

[1] https://github.com/srcdeps/srcdeps-maven
[2] https://github.com/srcdeps/srcdeps-gradle-plugin
[3] https://blog.gradle.org/introducing-source-dependencies


(Peter Palaga) #2

(4) Is there a way to define in the dependent project which gradle tasks should (or should not) be invoked to build the dependency?


(Daniel Lacasse) #3

Thanks @ppalaga for those great question and your contribution to the community with a solution for source dependencies. Here are the answer to your questions:

(1) There is no way to depend on a specific commit sha1, right? Are there any plans to add this?

At the moment, this is not possible. I created an issue to track this feature. It should be quite easy to add as a feature. What requires more thinking is how such feature will look in the user-facing DSL.

(2) Is there a way to depend on a Maven/Ant/sbt project?

You can add dependencies to any project Maven/Ant/sbt/etc. However, you need to provide a Settings plugin that bridges the project to Gradle. This sample shows how it can be achieved in the native world. The same applies to the JVM world.

(3) Knowing that source dependencies use composite builds under the hood, I am trying to figure out if the use of the artifacts produced by source dependencies is somehow limited. E.g. if I can include the dependency jar into a war. Or if I can get the jar and unzip it.

The implementation configuration cannot be resolved. For this purpose, you need to use either compileClasspath or runtimeClasspath depending on your intention. See this section of the user guide. The code would become:

task unzip(type: Copy) {
    from({ configurations.compileClasspath.asFileTree.collect {zipTree(it) } })
    into unzippedDir
}

(4) Is there a way to define in the dependent project which gradle tasks should (or should not) be invoked to build the dependency?

The Gradle tasks invoked to build the dependency are the one used when declaring the “exported” artifact. In the case of a Java project, the jar task and any of its dependencies will be executed. Any included build features should also work as expected.

Don’t hesitate to ask more questions,

Daniel


(Peter Palaga) #4

Thanks for your answer, @Daniel_L.

(1) Great!

(2) Interesting, will try to find some time to experiment.

(3) Thanks, compileClasspath works.

(4) Is there a way to define in the dependent project which gradle tasks should (or should not) be invoked to build the dependency?

The Gradle tasks invoked to build the dependency are the one used when declaring the “exported” artifact.

What is “exported” artifact. Is it something the dependent project has to declare explicitly?

In the case of a Java project, the jar task and any of its dependencies will be executed. Any included build features should also work as expected.

So if jar depends on running a long test suite, there is no way to avoid running the tests when building the source dependency?

Thanks again,

Peter


(Prasham Trivedi) #5

Not sure if it considered as a limitation or a question, but I would like to point it out.

How can I pass credentials to a private Git repository when defining it in source control?

As far as I have seen the docs, I can’t find anything which mentions how can I pass credentials to gitRepository url.

My Usecase: I am trying to build a dependency from a repository which is not Open source, but My user has rights to read that repository.


(Sterling Greene) #6

This is correct, but builds wouldn’t normally do this and I would recommend people to not do this. Regular multi-project builds behave in the same way.

At the moment, this isn’t really configurable. How do you access the private repo now? If it’s with SSH, I think this may work out of the box.


(Prasham Trivedi) #7

At the moment, this isn’t really configurable. How do you access the private repo now? If it’s with SSH, I think this may work out of the box.

At this point I am using HTTPs login. I will try SSH. But as a library developer I can’t force everyone to go on SSH route.

Later on I have discovered another Use case for Using Source Dependency.

I can use source dependency as a mechanism for distributing my private libraries.

Do you think this is a valid use case for that? Is it ok if I file an issue to allow passing credentials to private repository?


(Peter Palaga) #8

as a library developer I can’t force everyone to go on SSH route.

You cannot, but I think it is still still the best way to do that both for you and for the users or your lib.

I can use source dependency as a mechanism for distributing my private libraries. Do you think this is a valid use case for that?

It can work, but note that source dependencies work well only inside a Gradle build. They do not work well when there is a need to release the dependent project into a public Maven repository such as Maven Central. I hapen to have a picture for that. In the picture your user’s project is dependent-a and your lib is dependency-2.3.4-srcdeps. If dependent-a is released to a Maven repo, the dependency-2.3.4-srcdeps is not resolvable for any dependent-b that depends on your user’s project is dependent-a.

Source of the picture: http://ppalaga.github.io/presentations/181011-jcon-duesseldorf/index.html#/release_to_a_public_maven_repo_2


(Misha) #9

At the moment, this isn’t really configurable. How do you access the private repo now? If it’s with SSH, I think this may work out of the box.

It doesn’t. If I do this:

sourceControl {
    gitRepository("git@my.gitserver:user/project.git") {
        producesModule("org.example:project")
    }
}

then it fails with:
org.eclipse.jgit.errors.NoRemoteRepositoryException: file:///Users/myuser/work/git@my.gitserver:user/project.git: not found.


(Sterling Greene) #10

You have to translate this into a ssh:// URL: https://github.com/gradle/gradle-native/issues/839

This is a known thing we need to fix.


(Misha) #11

@sterling thanks for suggestion.
I changed it to gitRepository("ssh://git@my.gitserver/user/project.git") and it still doesn’t work. But now I have different exception:

Caused by: com.jcraft.jsch.JSchException: USERAUTH fail
        at com.jcraft.jsch.UserAuthPublicKey.start(UserAuthPublicKey.java:119)
        at com.jcraft.jsch.Session.connect(Session.java:470)

At the same time my git clone works perfectly. How can I tell jgit to get key from ~/.ssh/id_rsa?