# \[Proposal\] Task parallelization group

**URL:** <https://discuss.gradle.org/t/proposal-task-parallelization-group/43414>\
**Category:** Help/Discuss\
**Created:** [July 25, 2022, 12:31pm UTC](https://discuss.gradle.org/t/proposal-task-parallelization-group/43414 "2022-07-25T12:31:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![doublep](https://avatars.discourse-cdn.com/v4/letter/d/848f3c/32.png) [@doublep](https://discuss.gradle.org/u/doublep)\
**Post date:** [July 25, 2022, 12:31pm UTC](https://discuss.gradle.org/t/proposal-task-parallelization-group/43414/1 "2022-07-25T12:31:09Z")

</div>

I have searched on the internet to find out if it is possible to run tasks from a _single project build_ in parallel in Gradle. The answer appears to be “no”. Apparently there is some way to run _some_ work in parallel using `WorkerExecutor`, but this seems to be overly difficult and in any way not what I want.

I have come up with a simple proposal. Let Gradle tasks have _parallelization groups_. Those could be e.g. simple strings, though that’s up to Gradle developers to decide. As long as two (or more) tasks have _the same_ explicily set group, they can be run in parallel. Since it is fully up to the user to assign tasks to groups, any resulting failure can be blaimed onto the user. In other words, Gradle itself doesn’t have to provide tasks that are “safe” in this respect: it is the responsibility of the user to decide if in _his_ build given two tasks are independent from each other or not.

For example, given a build with tasks `minifyJs` and `minifyCss`, I would likely assign the two tasks to the same parallelization group like e.g. this:

```gradle
task ('minifyJs', type: ...) {
    parallelizationGroup = 'minifying'
}
task ('minifyCss', type: ...) {
    parallelizationGroup = 'minifying'
}

```

But in any case, Gradle wouldn’t impose anything on me: if I see the tasks are really independent, I do this, but if I don’t, by default Gradle simply always runs the tasks sequentially, as now.

Proposed Gradle behavior when deciding which task to run:

- build a set of tasks with dependencies (`dependsOn()`) already satisfied;
- if at least two tasks from the set have the same user-assigned parallelization group, run all (or at least some, subject to available CPUs, for example) those tasks in parallel;
- otherwise, i.e. if all tasks belong to different groups or have no groups set at all, behave as now.

Advantages:

- more control for the users: they know better what the tasks are supposed to do and thus can determine if they are independent or not;
- user is fully responsible for assigning parallelization groups, meaning that Gradle itself is not to blame if something is not correct (just like if a user forgets to add `dependsOn()` where needed);
- applicable to any standard task, tasks with `doLast()` and what not; doesn’t require custom tasks;
- requires minimal changes to `build.gradle`; likewise, requires minimal changes to disable this.

---

<div class="post-metadata">

**Author:** ![Vampire](https://sea1.discourse-cdn.com/gradle/user_avatar/discuss.gradle.org/vampire/32/9082_2.png) [@Vampire](https://discuss.gradle.org/u/Vampire)\
**Post date:** [August 17, 2022, 2:14pm UTC](https://discuss.gradle.org/t/proposal-task-parallelization-group/43414/2 "2022-08-17T14:14:29Z")

</div>

I cannot speak for the Gradle guys, and if you really want to suggest this, I’d recommend opening a feature request issue on GitHub instead, this is a community forum.  
But I don’t think this will be implemented, especially as with the upcoming configuration cache feature all tasks can be run in parallel if they don’t have an interdependency.

Btw. regarding

> just like if a user forgets to add `dependsOn()` where needed

whenever you add a manual `dependsOn()` (except when the left-hand side is a lifecycle task) you are most probably adding a code smell.  
You should usually wire task outputs to task inputs or other places where implicit task dependencies are understood and by this automatically get the necessary task dependency for all required cases.

Just a small example, if you have a task that generates some code and then register the path manually as source path and adding a manual dependency from the compile task, to the generate task, you immediately have an invalid build. Any other place that needs sources will either miss the generated sources if the generate task didn’t happen to run before, or get stale sources if the generate task run before but not in the same invocation but previously. If you instead properly register the task output (or if all outputs are fine the task itself) as source directory, then it is automatically a dependency for all consumers of the source files like JavaDoc task, sources jar task, …