Change of behaviour for @Finalize between 2.3 and 2.4 release

Francois_Guillot · May 9, 2015, 8:54am

In my custom Gradle plugin I have the following logic, which is working OK when executed on a Gradle 2.3 distribution:

apply the ivy-publish plugin
automatically add some repositories to the ivy publish plugin repositories (repA, repB)
declare some publications in the build.gradle script where my plugin is applied (let’s say pubA, pubB)
the ivy publish plugin will create tasks publishPubAToRepA, publishPubBToRepA, publishPubAToRepB, publishPubBToRepB
declare custom tasks in my plugin (customA, customB)
Use a @RuleSource with @Finalize method to finalize my task by the regular tasks from the ivy publish plugin

@Finalize
public void addFinalizedTasks(TaskContainer tasks){
tasks.withType(CustomATask).each {CustomATask myTask->
logger.debug(“finalize ${myTask.name}”)
tasks.withType(PublishToIvyRepository).matching { PublishToIvyRepository origTask ->
origTask.publication.name==myTask.theOriginalPublication.name && origTask.repository.getName()==myTask.repo.getName()
}.each {
myTask.finalizedBy it
}
}
}

Note that I add to do it in a @Finalize method, instead of a @Mutate method on the task container, because for this to work, the ivy publish plugin must have created its own tasks.

Anyway, this was working OK in 2.3.

When switching to 2.4, I recompiled my plugin, replacing the @RuleSource by extends RuleSource. That is the only thing I did.

And now, my custom tasks are not finalized by the original ivy plugin tasks.
Does anybody have a hint on what has changed between 2.3 and 2.4, that could lead to this behaviour ?

Francois_Guillot · May 9, 2015, 9:08am

Here are some extracts from my debug log

my custom tasks are well created

10:57:32.327 [DEBUG] [org.gradle.api.Task] creating publishIvyRelease
10:57:32.342 [DEBUG] [org.gradle.api.Task] creating publishIvyShared
10:57:32.358 [DEBUG] [org.gradle.api.Task] creating publishIvyWindows

the ivy publish @Mutate is called

10:57:33.170 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Mutating tasks using org.gradle.api.publish.ivy.plugins.IvyPublishPlugin$Rules#createTasks(org.gradle.model.collection.CollectionBuilder<org.gradle.api.Task>, org.gradle.api.publish.PublishingExtension, java.io.File)
10:57:33.186 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Transitioning model element ‘tasks.generateDescriptorFileForIvyPublication’ from state Known to Initialized
10:57:33.201 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Transitioning model element ‘tasks.__instantiator’ from state GraphClosed to GraphClosed
10:57:33.201 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Transitioning model element ‘tasks.__instantiator’ from state GraphClosed to GraphClosed
10:57:33.217 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Creating tasks.generateDescriptorFileForIvyPublication using org.gradle.api.publish.ivy.plugins.IvyPublishPlugin$Rules#createTasks(org.gradle.model.collection.CollectionBuilder<org.gradle.api.Task>, org.gradle.api.publish.PublishingExtension, java.io.File) > create(generateDescriptorFileForIvyPublication)
10:57:33.233 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Mutating tasks.generateDescriptorFileForIvyPublication using org.gradle.api.publish.ivy.plugins.IvyPublishPlugin$Rules#createTasks(org.gradle.model.collection.CollectionBuilder<org.gradle.api.Task>, org.gradle.api.publish.PublishingExtension, java.io.File) > create(generateDescriptorFileForIvyPublication)
10:57:33.248 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Transitioning model element ‘tasks.__store’ from state Known to GraphClosed
10:57:33.264 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Creating tasks.__store using Project..tasks.__store()
10:57:33.264 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Finished transitioning model element tasks.__store from state Known to GraphClosed
10:57:33.279 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Transitioning model element ‘tasks.__store’ from state GraphClosed to GraphClosed
10:57:33.279 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Mutating tasks.generateDescriptorFileForIvyPublication using DefaultCollectionBuilder.createAndStoreVia() - tasks.generateDescriptorFileForIvyPublication
10:57:33.295 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Finished transitioning model element tasks.generateDescriptorFileForIvyPublication from state Known to Initialized
10:57:33.295 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Transitioning model element ‘tasks.publish’ from state Known to Initialized

My @Finalize method is called

10:57:33.389 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Mutating tasks using com.xxxx.gradle.plugins.jfdp.JfdpPlugin$Rule#addFinalizedTasks(org.gradle.api.tasks.TaskContainer)
10:57:33.404 [DEBUG] [com.xxxx.gradle.plugins.jfdp.JfdpPlugin] finalize publishIvyRelease
10:57:33.404 [DEBUG] [com.xxxx.gradle.plugins.jfdp.JfdpPlugin] finalize publishIvyShared
10:57:33.420 [DEBUG] [com.xxxx.gradle.plugins.jfdp.JfdpPlugin] finalize publishIvyWindows
10:57:33.451 [DEBUG] [org.gradle.model.internal.registry.DefaultModelRegistry] Finished transitioning model element tasks from state Created to SelfClosed

It seems that the ivy tasks are not in the task container when my @Finalize method is called (otherwise the wiring between my custom tasks and the ivy regular tasks would have occurred)

Francois_Guillot · May 9, 2015, 9:26am

Mmmm, if I add logger.warn(tasks.toString()) at the beginning of my @Finalize method, I don’t see the regular ivy publish tasks publishPubAToRepA, publishPubBToRepA, publishPubAToRepB, publishPubBToRepB

This is strange.

As explained above, the publications are added in the consuming build.gradle file
publishing {
publications {
ivy(IvyPublication) {
from components.java
organisation “$myOrganisation”
module = “$myModule”
}
}
}

and the repositories are added by my plugin, from the project repositories:

         project.repositories.all { repo ->
            if (repo instanceof IvyArtifactRepository) {
                project.getExtensions().configure(PublishingExtension, new ClosureBackedAction<PublishingExtension>({ repositories.add(repo) }))
            }
        }

Francois_Guillot · May 9, 2015, 2:38pm

I don’t know where the bug lies, but I managed to produce a SSCCE showing the problem

Everything is in this Gist https://gist.github.com/facewindu/70a91a63e563a66f9809 :
Gradle 2.3 build file + output
Gradle 2.4 build file (with only @RuleSource replaced by extends RuleSource) + output

When executing customTask, I expect the publishIvyToIvyRepository task to be added after

Francois_Guillot · May 11, 2015, 11:22am

In the 2.4 release note, in the ‘Potential Breaking Changes’ section, I see

The withType() method selects elements based on the public contract type rather than implementation type.

This might be the reason for my problem, but then I’m not sure how to understand this change of behavior and how to modify my RuleSource to cope with this change.

luke_daley · May 12, 2015, 2:41am

The basic problem here is that Gradle no longer creates all tasks. Part of the rules based mechanism is the ability to not create something unless it is needed. Where it’s falling down here is that the constructs you are using don’t actually express that it is needed. Unfortunately, there’s no way right now to express that all CustomTask objects depend on all PublishToIvyRepository tasks. That’s what you really need. It’s coming.

Your options for right now are to:

Call tasks.realize() at the end of your script to force the creation of all tasks (please note: this is an internal method)
Use naming conventions to wire things together

Example…

	static class Rule extends RuleSource{
		@Finalize
		public void addFinalizeOnCustomTask(org.gradle.api.tasks.TaskContainer tasks){
			tasks.withType (CustomTask) {myTask ->
				myTask.finalizedBy { tasks.names.findAll { it.startsWith "publishIvy" } }
			}
		}
	}

Francois_Guillot · May 12, 2015, 6:17am

@luke_daley: thanks for your precise answer

Do you have a rough idea of the when ? (like one or two releases, or definitely not before 2016 …)

In the meantime, I think I will stick to 2.3.
I’m not a big fan of using hard-coded naming conventions for my build logic, and I don’t know the side effects that calling tasks.realize() can occur (and anyway, it’s not a real, lasting, solution)

One more thing:
If I understand you correctly, when the missing construct we’re talking about will be available, I will be able to create a rule directly between the tasks of type CustomTask and the tasks of type PublishToIvyRepository, without using the whole ModelMap<Task> as the rule subject ? If true, does this mean that I will be able to switch from @Finalize (required in 2.3 to be sure that the PublishToIvyRepository tasks were created) to @Mutate, right ?

luke_daley · May 12, 2015, 6:29am

Very high likelihood of one or two releases. We need this construct for other cases too.

All correct.

Francois_Guillot · May 12, 2015, 6:43am

cool !
I already love the rule based model, I can’t wait to see more of what it has in its guts. (Plus it’s fun to use )

Francois_Guillot · June 27, 2015, 10:53am

@luke_daley
any news on that ?

luke_daley · July 1, 2015, 2:01am

Not at this time, no.

Volkman · August 20, 2015, 1:54pm

@luke_daley

Any news on that in 2.6?

My problem is described here.

Francois_Guillot · August 20, 2015, 3:01pm

The specs to get it fixed has just been written
See https://groups.google.com/forum/m/#!topic/gradle-dev/y9WmHyzCX-s
I don’t know if this will make it to 2.7 or if it’s gonna be later

Adrian_Kelly · August 20, 2015, 3:37pm

Most likely it will be 2.8.

Jim_Gish · August 24, 2015, 9:43pm

We have the same issue, so we’re eager for the fix as well. Thank you.

Topic		Replies	Views
Ordering between @RuleSource/@Mutate calls Help/Discuss	2	609	April 3, 2015
Add project repos to Publising repos using Rule Source Help/Discuss	1	1440	May 12, 2015
Add dependency to GenerateIvyDescriptor task Help/Discuss	0	417	September 22, 2015
Ivy publish task not created when a dependency of a dynamically created task Help/Discuss	2	863	December 9, 2015
Modifying version number and publishing with new publishing plugin Old Forum Archive	2	2561	August 9, 2013

Change of behaviour for @Finalize between 2.3 and 2.4 release

Related topics