Project dependencies ignored when compiling custom source set

A version of this was posted yesterday on Stack Overflow: https://stackoverflow.com/q/62474238/1151724

I have a small library project in kotlin with a custom source set that I use for demos. Each demo is a single file with a main(), included in a fat executable jar. I prefer to keep them all in the same directory, separate from the main and test sources, which don’t require them (and nothing requires more than one of them).

My build.gradle.kts looks like this; I think not all of it is relevant to the problem (but it is not too long). The demo jar tasks are created in a loop at the bottom:

plugins {
    kotlin("jvm") version "1.3.72"
}

repositories { mavenCentral() }

dependencies {
    implementation(kotlin("stdlib-jdk8"))
	testImplementation("org.jetbrains.kotlin:kotlin-test")
	testImplementation("org.jetbrains.kotlin:kotlin-test-junit")
}

// Yes, I know the source paths are non-standard.  These are three separate toplevel directories.
sourceSets["main"].java.srcDir("src")
sourceSets["test"].java.srcDirs("test")
sourceSets {
	create("demo")?.let {
		it.java.srcDir("demo")
		it.compileClasspath += main.get().output
		it.runtimeClasspath += main.get().output
	}
}

tasks {
	withType<Test> {
		this.testLogging { this.showStandardStreams = true }
	}
	withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile> {
		this.kotlinOptions.jvmTarget = "1.8"
	}
}

// Demo jars.
listOf("InteractiveClient", "LockingBufferDemo").forEach {
	tasks.register<Jar>(it) {
		manifest { attributes["Main-Class"] = "${it}Kt" }
		from(sourceSets.main.get().output)
		from(sourceSets["demo"].output) {
			include("**/${it}Kt.class")
		}
		dependsOn(configurations.runtimeClasspath)
		from({
			configurations.runtimeClasspath.get().filter {
				it.name.endsWith("jar") }.map { zipTree(it) }
		})
	}

}

Here’s the problem: When I try to build any of the jar targets, I get the long list of errors indicative of the kotlin stdlib not being available (“Cannot access built-in declaration ‘kotlin.String’. Ensure that you have a dependency on the Kotlin standard library” etc.).

If I change this slightly by removing the “demo” sourceSet declaration and changing from(sourceSets["demo"].output) to from(sourceSets["test"].output) in the jar task definition, plus move the demo sources into the test directory, everything works as expected and desired.

So, the dedicated demo source set is not completely necessary, but it does help organise things meaningfully, etc. I presume that since it is possible to create more than two source sets, it should be possible to compile them much the same way as the built-ins.

When the demo source set is created, a number of configurations are also created (demoImplementation, demoRuntimeOnly, see Java Plugin docs for full list). While you are setting demo's compile and runtime classpaths to include the compiled output of the main source set, you are not configuring the compile or runtime classpath dependencies using the corresponding demo* configurations. If you want demo's configurations to mirror the main source set’s, then use the extendsFrom method of the configurations. For example, configurations.demoImplementation.extendsFrom(configurations.implementation).