How do I compile multiple sourceSets together?

Old Forum Archive
1

The architecture of our system is a little messy. We have a big ball of mud that consists of ~8 clearly different components which, unfortunately, all have to be compiled together. (There are lots of hairy dependency cycles. We want to fix them, but we want a sane build system first.)

I’d like to be able to specify those 8 distinct components as sourceSets, even though they do not compile independently (yet). If I have to specify one giant sourceSet for the whole ball-of-mud, then 1) it’ll be huge and 2) it’ll be hard to remember where we want to draw the lines between those 8 components when it comes time to fix the dependency cycles.

Any suggestions?

2

My general feeling is that this will create another ball of mud. Without knowing your specific situation, I think I’d rather have just one source set and draw the component boundaries by structuring the build script(s) accordingly.

3

Yeah, that’s kind of what I thought. I’ve almost got it working like this:

component1_srcdirs = [...]
component1_classpath = [...]
  componentN_srcdirs = [...]
componentN_classpath = [...]
  sourceSets {
    mudball {
        java {
            srcDirs = component1_srcdirs + ... + componentN_srcdirs;
        }
        compileClasspath = files(component1_classpath + ... + componentN_classpath);
    }
}

But I’m having trouble with the exclusions. You see, for most of those components, I want to exclude ‘**/Mock*.java’, but for one particular component I need to include those files.

Attempt #1:

mudball {
         java {
             srcDirs = ...
            exclude('**/Mock*.java')
            include('component5/**/Mock*.java')
        }
    }

But that resulted in an empty sourceSet, which made perfect sense once I read the fine documentation for PatternFilterable (which sset.java is an instance of). OK, fair enough.

Attempt #2:

mudball {
         java {
             srcDirs = ...
            exclude('component3/**/Mock*.java')
            exclude('component7/**/Mock*.java')
        }
    }

(Since component3 and component7 are actually the only ones with any Mock*.java files to exclude; in principal I should list all components except component5 here, but I’m lazy.)

That just plain does not work.The 3 source files that match those two exclusion patterns are included in the sourceSet anyways, making the build fail.

Attempt #3:

mudball {
         java {
             srcDirs = ...
            exclude('/home/gward/src/ourproduct/component3/source/com/example/comp3/MockThing.java')
            exclude('/home/gward/src/ourproduct/component7/source/com/example/comp7/MockFoo.java')
            exclude('/home/gward/src/ourproduct/component7/source/com/example/comp7/MockBar.java')
        }
    }

Obviously this isn’t “real”, but I wanted to see if maybe there was a wildcard problem. But this still doesn’t work; Gradle tries to compile those three files regardless.

Attempt #4:

mudball {
         java {
             srcDirs = ...
            exclude('**/MockThing.java')
            exclude('**/MockFoo.java')
            exclude('**/MockBar.java')
        }
    }

Still no luck.

Any ideas?

4

1 and 2 don’t match the full source file path, but 3 and 4 look fine. Without a reproducible example, it’s hard for me to say what the problem is. Maybe the files are getting compiled by a compile task other than compileMudJava ( e.g. compileTestJava)?

5

I also have a lot of different components that need to be built independently in order.

I’ve been able to do

gradle build nameOfSourceSet

and I see the output of a single one built inside build/nameOfSourceSet.

but how do I tell gradle to build all of them on a task?

so I can build the whole project, and then move on to creating a single jar that encompasses all my classes.

sourceSets {
      //common/components
    apache {
        java {
            srcDir 'common/components/apache'
        }
    }
      gson {
        java {
            srcDir 'common/components/gson'
        }
    }
      httpserver {
        java {
            srcDir 'common/components/httpserver'
        }
    }
      jdownloader {
        java {
            srcDir 'common/components/jdownloader'
        }
    }
      jdmns {
        java {
            srcDir 'common/components/jdmns'
        }
    }
      json {
        java {
            srcDir 'common/components/json'
        }
    }
      minicastle {
        java {
            srcDir 'common/components/minicastle'
        }
    }
      mp4parser {
        java {
            srcDir 'common/components/mp4parser'
        }
    }
      okhttp {
        java {
            srcDir 'common/components/okhttp'
        }
    }
      vuze {
        java {
            srcDir 'common/components/vuze'
            exclude '**/bouncycastle/**'
                        exclude '**/jman/**'
            exclude '**/swing/**'
            exclude '**/swt/**'
            exclude '**/systray/**'
            exclude '**/tests/**'
        }
    }
      commoncore {
        java {
            srcDir 'common/components/core'
        }
    }
      //now components
      setting {
        java {
            srcDir 'components/setting/src'
        }
    }
      common {
        java {
            srcDir 'components/common/src'
        }
    }
      resources {
        resources {
            srcDir 'components/resources/src/main/resources'
        }
    }
      alexandria {
        java {
            srcDir 'components/alexandria/src'
        }
    }
      osx_stub {
        java {
            srcDir 'components/osx_stub/src'
        }
    }
      //now core
      core {
        java {
            srcDir 'core'
        }
    }
      //finally gui
      gui {
        java {
            srcDir 'gui'
        }
    }
}
6

This should be done as a mutli-module build, not a single module as you are attempting. Each ‘component’ should be in a separate module. You can have dependencies between the modules and gradle will build the individual modules (jars) in the correct order.

Once you have the individual modules building their separate jars, you can create an uber-jar module which builds a jar for all the components. This can be done using shadow

https://github.com/johnrengelman/shadow