Disabling parallel for certain tasks only


(Derek Ealy) #1

Hi,

We use the org.gradle.parallel=true pretty heavily in our project. There are certain tasks however that usually fail when run in parallel mode. Is there any way to specify that some tasks are never to be run in parallel? I know I can set -Dorg.gradle.parallel=false when executing my task by itself but there are times when I’d like to run several tasks together and for the system to just know not to execute certain ones in parallel.

This usually has to do when installing or deleting multiple APKs.

Thanks, Derek


(Sterling Greene) #2

I think we check if tasks have overlapping output directories and won’t run them in parallel if they do. So you could try having a dummy output directory that’s shared by all tasks that shouldn’t run in parallel. We also won’t run tasks that are in the same project in parallel (this will eventually change).

Setting org.gradle.parallel=false after task execution has started won’t turn off parallel execution.

You could also look at the mustRunAfter methods on Task. You could then setup the tasks to run in a particular order, but not require all of them to run (like dependsOn would).


(Derek Ealy) #3

Awesome, the common output directory works. Thanks for the idea.


(jim.gish) #4

This is an example of a “cool”, non-standard, trick that ultimately reads “hack” and makes builds hard to understand and maintain over time. It would be far preferable to have some dynamic equivalent of parallel=false that. Please consider that.


(Sterling Greene) #5

Yeah, this is in “hack” territory. ‘mustRunAfter’ would still be hacky, but at least it would make it clear there’s an ordering that could be documented (vs an output that’s never used).

Just having a “don’t run this task in parallel” isn’t quite what we’d want long term. We’d like to be able to say there’s some common resource and that’s an input/output of the task and Gradle understands that some tasks need to wait for that dependency to become available.


(jim.gish) #6

This is a perfect example of the overall philosophy of gradle that I’d like to see tempered a bit in some cases. The usual gradle model is one where it infers & computes the proper build behavior on the basis of task dependencies, inputs, outputs, etc. This is a very good thing, especially given the complexity of large builds, where no one person has the knowledge or ability to comprehend intricate project & build inter-dependencies. However, there are times where the developer working in a small part of the build simply knows that there are build constraints and needs a straightforward and clear way aside from the normal artifact/task-based model of constraining the build in some specific way, such as parallel=false, which overrides the model-based behavior.


(Richard) #7

I found a use case where parallel=false in a task makes sense to me.

I want to list all the class paths of all my sub-projects.

Right now because I want to build in parallel I get my class list intertwined. How do I make this work properly (i.e. the way I want it to :wink: )?

subproject {
    task printJavaPaths << {
      println "${project} mainClasspath:"
      sourceSets.main.runtimeClasspath.each { File file ->
        println "      ${file}"
      }
}

(or at least the ability to switch parallel off in the command line, it seems i can switch parallel on from the command line, but not off)


(Vicky Sirwani) #8

I tried mustRunAfter in a multi module project for the build tasks of the modules, however I am not able to use that. My usecase is that I want all of my modules to build parallelly except for one which I want to build after all the other modules have been built parallely.

It would be helpful if you could suggest something for this.

Thanks.


(Angel Taveras) #9

I was looking for something like above. On running some tasks in parallel and some serially. I have tests that binds to a port and if two of those tests runs at the same time one of them will fail. Any ideas?


(Jason Griffith) #10

My guess would be to either explicitly say that whatever tasks contend for that port need to “.mustRunAfter” each other, or set those tasks outputs to an overlapping location so that Gradle won’t run them in-parallel.

We did the first option when we found that a third party app that claimed parallelism support actually didn’t, so we programmatically found those tasks by name and set them to run after each other in a line.


(Andreas Axelsson) #11

I have another use case for disabling parallel execution. We have a tool which simply refuses to work if you start two instances of it, but the order in which two tasks are run is otherwise irrelevant.

A related reason is that the tool in question will use all available cores on a machine for a single process, so running it in parallel wouldn’t make help, even if it worked.

It’s also a composite build, where having to configure .mustRunAfter on arbitrary sub-tasks becomes a bit awkward.

So in my case, I’d like to say that a task of a certain type must not run at the same time as another task of that type, or subtype.


(Andreas Axelsson) #12

And in gradle 4.5 using --no-parallel doesn’t seem to disable parallel execution for a composite build either.


(Ross Goldberg) #13

--no-parallel didn’t seem to disable parallel composite builds in Gradle 4.8.1.

Is Gradle currently supposed to be able to be forced to build composite builds serially instead of in parallel?

  • If so, then what is the proper way to do so?
    • Is that methodology currently broken by a bug? If so, is there a workaround?
  • If not, is there a GitHub issue to add this functionality?