Parallelize build : Where to start

Help/Discuss
1

Hello All,

New here. New to gradle as well.

I have a project build which I would want to parallelize.
I looked up decoupled projects requirements : https://docs.gradle.org/current/userguide/multi_project_builds.html#sec:decoupled_projects

Couple of things are not clear yet:

  • Does configuration injection make projects coupled? Because in the forums people say that configuration injection is okay because it is at configuration phase. Which is correct ?

  • Is there a way in which I can check the couplings which are existing?

  • As far as my understanding, to run build tasks in parallel, you either run it with --parallel OR change your gradle.properties to reflect org.gradle.parallel=true

  • Is there any way to parallelize only some tasks?

Thanks in advance,
Dhaval

2

Hi Dhaval,

great to have you on board :slight_smile:

There are two kinds of coupling:

  • coupling at configuration time (e.g. having a subprojects {} block): This has no effect on --parallel execution, but it interferes with --configure-on-demand
  • coupling at execution time (some task changes the configuration of another project or accesses an unsynchronized shared resource): This will break --parallel execution.

You can use either or both, it does not matter :slight_smile:

No there is not. Do you have a specific use case in mind?

Cheers,
Stefan

3

I have a main project. Which builds 3 libs, they shouldn’t be parallelized. Then after I have 4-5 android samples which uses those libs, I would just like to parallelize them (obivously, the number is going to keep on increasing from 4-5)

4

Why? It doesn’t hurt :slight_smile:

5

I think they modify each other at runtime. That’s the problem.

6

Hey Buddy,

Somehow I feel dependencies are not working properly for me.

I have project like this(this is minimal):

  • Jar Libs

  • Module-Core

  • Module-Balloon

build.gradle for Module-Core:

android{
tasks.withType(JavaCompile) { compileTask ->
if (compileTask.name.startsWith(“compileDebugJava”)) compileTask.dependsOn buildNativeDebug
if (compileTask.name.startsWith(“compileReleaseJava”)) compileTask.dependsOn buildNativeRelease
}
}
dependencies {
compile fileTree(dir: ‘libs’, include: [’*.jar’])
}

.
.

build.gradle for Module-Balloon:

android{

tasks.withType(JavaCompile) { compileTask ->
if (compileTask.name.startsWith(“compileDebugJava”)) compileTask.dependsOn buildNativeDebug
if (compileTask.name.startsWith(“compileReleaseJava”)) compileTask.dependsOn buildNativeRelease
}
clean.dependsOn ‘cleanNative’
}

dependencies {
compile project(’:libraries:core’)
}

When I run gradle build into main project, these 2 tries to run in parallel

  • balloon:buildNativeDebug

  • core:buildNativeRelease

and fails the build.

Haven’t I declared already that ‘Moudle-Core’ should be compiled first before compiling ‘Module-Balloon’ ?

Could you throw some light?

Thanks a ton in advance,
Dhaval

7

Why and how are they doing that? Can you give an example? If this cannot be avoided, you’ll have to build your own locking mechanism for those tasks to prevent them from running in parallel.

The build is broken down by tasks, not projects. The buildNativeRelease task of core does not depend on the buildNativeDebug build of baloon. They can be run in parallel.

8

The case with this is that they try to access particular resource at the same time and eventually fail:

  • buildNativeRelease and buildNativeDebug of each Module tries to access the same resource

Is there any easy way to manage locking of resources ? OR I will have to write my own?

9

You’ll have to roll your own in that case.

Just curious: What kind of shared resources are these tasks accessing?

10

It is a lib from 3rd party firm.