Defining Consumer from Groovy build.gradle

I have an extension object that collects a number of Consumer references:

class ProfileDefinition {
    ...
    private ValueContainer<Consumer<TestDescriptor>> beforeEachTestActions = ...;
}

class ValueContainer<T> {
    private List<T> values;

    ...

    public void addValue(T value) {
        if ( values == null ) {
            values = new ArrayList<>();
        }
        values.add( value );
    }

}

The only way I have been successful so far in configuring this setting is:

beforeEachTestActions.addValue(
            new Consumer<TestDescriptor>() {
                	@Override
                    void accept(TestDescriptor testDescriptor) {
                        printf( "Executing test `%s` against custom profile", testDescriptor );
                    }
            }
)

I’d like to avoid the explicit new Consumer<TestDescriptor>() creation. Any idea how to do that?

This should work with a normal Groovy closure:

beforeEachTestActions.addValue { testDescriptor ->
    printf("Executing test `%s` against custom profile", testDescriptor);
}

Java Lamda syntax isn’t supported in the Groovy compiler, so you can’t really write anything that’s more Java-esque here short of the long version you started with.

Thanks James. Still having a strange problem with that syntax. Seems odd, but I cannot figure it out…

class ProfileDefinition {
    ...
    private ValueContainer<Consumer<TestDescriptor>> beforeEachTestActions = ...;
    private ValueContainer<Consumer<Test>> beforeTestTaskActions = ...;
}

-----------------

// this one parses fine

beforeEachTestActions.addValue { testDescriptor ->
    printf( "Executing test `%s` against h2 profile", testDescriptor );
}

// this one does not:
//     Caused by: java.lang.ClassCastException: class profile_6f0q1sabc0rjv7o99xt63milb$_run_closure2 cannot be cast to class java.util.function.Consumer

beforeTestTaskActions.addValue { task ->
    printf( "Executing Test task `%s` against h2 profile", task );
}

If I remove the second one, everything is fine. Something to do with it being a task? FWIW, I originally tried Action but had the same problem.

I’m filling in some gaps here, but I can’t replicate that error when throwing that code in a rudimentary project and adding some things so it will run. I can add values with the Closure to both ValueContainers, so there’s some critical difference in your real project that isn’t clear to me yet.

Thanks again James. I’ll work on getting it pushed. It is a plugin, I do not have any “real” project uses - this is all via GradleRunner / testkit.

The only real “odd” thing I can think of in the plugin code is the use of Project#apply(Map) after evaluation. Though if that is the problem, I would think that all blocks would have this issue

I think I solved this. As far as I can tell the problem was that I attempted to have ProfileDefinition also be a Binding. I was playing with how to expose this to the “Gradle fragment” I apply and just tried playing with that for no real good reason :wink:

Anyway, I removed that and it seems to work pretty consistently now.

Relatedly, any pointers to resources that discuss how to expose beans / dsl-extensions to gradle fragments? Something like project.getExtensions().create(...) but that I can later remove?

The basic idea is that the plugin finds a number of “database profiles” used for testing. Each profile is defined using a gradle fragment. At them moment I apply each fragment ((GradleProfileFragment == ProfileDefinition) as:

	final GradleProfileFragment profileFragment = new GradleProfileFragment( defaultProfileName );

	project.getConvention().getPlugins().put( PROFILE_KEY, profileFragment );

	try {
		project.apply( Collections.singletonMap( "from", profileFile ) );
	}
	finally {
		project.getConvention().getPlugins().remove( PROFILE_KEY );
	}

So that profile descriptor should only be in effect while processing that single file.

What I do not understand is that PROFILE_KEY is "profile", but I cannot specify profile {} in the fragment. I have to use the unqualified attribute names. E.g., this works:

beforeEachTest {
    ...
}

However, this does not:

profile {
    beforeEachTest {
        ...
    }
}

How would I make that work? Can I add it to that Map passed to Project#apply? Another way?

Thanks again James

This is a difference between a convention and an extension. The convention essentially merges the methods of one object directly into the other. In a typical Java project, things like source sets and source/target compatibility are part of the Java plugin convention, but are used directly in build.gradle:

sourceCompatibility = '1.8'
targetCompatibility = '1.8'

sourceSets {
    main {
        java {
            srcDir 'src/generated/java'
        }
    }
}

In contrast, extensions are nested under the extension name. If you wanted the same for a convention, you would have to do the nesting of another object yourself. It’s also been said that extensions are expected to fully replace conventions eventually, but this hasn’t been a very quick process.

I don’t think you should actually be trying to remove anything. I think the concept of the fragment should be implemented completely independently of the DSL using some of the concepts in Custom Gradle Types. Imagine a DSL like this:

databaseProfiles {
    h2Profile {
        beforeEachTestAction { testDescriptor ->
            printf("Executing test `%s` against h2 profile", testDescriptor)
        }
        beforeTestTaskAction { task ->
            printf("Executing Test task `%s` against h2 profile", task)
        }
    }
    otherDatabaseProfile {
        beforeEachTestAction { testDescriptor ->
            printf("Executing test `%s` against other profile", testDescriptor)
        }
        beforeTestTaskAction { task ->
            printf("Executing Test task `%s` against other profile", task)
        }
    }
}

Nothing needs to be removed here. The configuration is contained to the specific ProfileDefinition that is being configured, so you don’t need to worry about even what’s in a particular file. Code can independently be written that would find fragments and add them to the databaseProfiles extension separately, perhaps with the filename being the name in that case. If you don’t want them to be named, that’s fine too. It just changes the methods called slightly.

Thanks again James. I really appreciate your help. This is not a simple series of questions, but your answers are really helpful.

But I agree with you 100% that that is the DSL I like conceptually.

I had not thought of extending the extension (I already have a main DSL extension object) to include these profiles and dynamically adding discovered ones. This would be a “named container” I assume?

So my existing DSL extension looks like, e.g.:

databases {
    defaultProfile = 'derby'

    profileDirectory 'custom-databases'

    ...
}

Like I said, I conceptually like the idea of adding the profiles to this DSL:

databases {
    defaultProfile = 'derby'

    profileDirectory 'custom-databases'

    profiles {
        ...
    }

    ...
}

I’m just not sure about the specifics of:

find fragments and add them to the databaseProfiles extension separately

I’m sure I am missing something.

So I assume you mean is something like:

public class DslExtension {
    private final Project project;

    private String defaultProfile = "h2";
    private final List<File> profileDirectories = new ArrayList<>();
    ...
    /** 
     * Think of Profile as GradleProfileFragment.  ATM we allow 2 distinct ways to define a profile:
     *    1) via a Gradle fragment
     *    2) via a dir with properties file and jdbc jars
    *
    * But for making this easier... let's assume there is just one way - via a Gradle fragment and
    * that Profile == GradleProfileFragment
    */
    NamedDomainObjectContainer<Profile> profiles;
}

So when Gradle sees, e.g.

databases {
    ...
    profiles {
        h2 {...}
        derby {...}
    }
}

it will automatically create 2 profiles in the DslExtension#profiles NamedDomainObjectContainer named h2 and derby. But I am still unsure of how to process the Gradle fragment files I discover.

Is the idea that the Gradle fragment file would specify that full “path”? I.e.:

profile.gradle
---------------
databases {
    profiles {
        oracle {...}
    }
}

That’s not ideal, but if it works I can live with that.

Am I understanding you correctly?

With what I was mentioning, you now have 3 ways to define a profile.

Inline:

databases {
    ...
    profiles {
        h2 {
            beforeEachTestAction { testDescriptor ->
                printf("Executing test `%s` against h2 profile", testDescriptor)
            }
            beforeTestTaskAction { task ->
                printf("Executing Test task `%s` against h2 profile", task)
            }
        }
        derby { ... }
    }
}

Fragments:

// h2.gradle
h2 {
    beforeEachTestAction { testDescriptor ->
        printf("Executing test `%s` against h2 profile", testDescriptor)
    }
    beforeTestTaskAction { task ->
        printf("Executing Test task `%s` against h2 profile", task)
    }
}
// plugin code

project.files(profileDirectory).filter { include '*.gradle' }.each { fragment ->
    project.apply(from: fragment, to: databases.profiles)
}

This is really rough and may not compile as is. Essentially you need code that can apply your fragment to what you have that can handle it. You can tweak this code though. Maybe a generic profile container in the fragment that corresponds to a method on databases different than the NamedDomainObjectContainer, but that properly adds it.

Then you have the code that can look up the properties and manually add them to the NamedDomainObjectContainer. That’s essentially read property file, map to ProfileDefinition, add to NamedDomainObjectContainer like you would any collection.

project.files(profileDirectory).filter { include '*.gradle' }.each { fragment ->
    project.apply(from: fragment, to: databases.profiles)
}

Ah, this was the piece I was missing - the to:.

I actually made all of the changes related to using NamedDomainObjectContainer for collecting the profiles and it worked like a champ. The part that I was missing was that you can Project#apply to something (other than the project). I’ll look at that next.

Thanks again!! :slight_smile:

I tried adding to: to the #apply call, but that did not work for me.

if ( pathName.endsWith( ".profile" ) || pathName.endsWith( ".gradle" ) ) {
    project.apply( Helper.asMap( "from", path, "to", dslExtension.getProfiles() ) );
    ...
}

and in one of these files:

h2 {
    hibernateProperty ...
    dependency 'com.h2database:h2:1.4.196'
    ...
}

But end up with:

* What went wrong:
A problem occurred evaluating script.
> Could not find method h2() for arguments [profile_3zuremxljk3s5vzzp1lhlqr9o$_run_closure1@59a2c35d] on object of type org.hibernate.testing.db.Profile.

So it seems like passing the NamedDomainObjectContainer as the to target somehow thinks the NamedDomainObjectContainer is one of the contained domain objects.

Am I just doing something wrong?

It does work if do not specify to and have the fragments do the full databases/profiles/h2 naming. Meaning, this works:

databases {
    profiles {
        h2 {
            ...
        }
    }
}

Were you expecting that I’d pass something other than the NamedDomainObjectContainer as to?

Yeah, sorry. That version was really rough. You can’t do exactly that because there’s a limitation that whatever you apply the fragment to: needs to have something that can receive the top level item in the fragment. In this case, the NamedDomainObjectContainer doesn’t have a method/property called h2, so it fails.

We can work around this in several ways, but the question is going to be which one makes the most sense for your requirements. While you can certainly have each fragment contain the entire databases {} block, you can do at least one better. Here’s some options. Hopefully, it’s clear what’s what as I modify some working code to align with code and naming from your examples.


#1. Remove one level from the fragment and apply to databases.

// plugin
project.fileTree(profileDirectory).matching(files -> files.include("**/*.gradle")).forEach(file -> {
    project.apply(Helper.asMap("from", file, "to", dslExtension));
});

You include profiles {}, but not databases {} in the fragment:

// h2.gradle
profiles {
    h2 {
        ...
    }
}

#2. Add a helper method in the extension for adding from the fragment.

The plugin code stays the same, and you apply to: the dslExtension, but you add a method to the extension that will end up executing the fragment code against a newly created :

void profile(String name, Action<DatabaseProfile> profileAction) {
    DatabaseProfile profile = new DatabaseProfile(name);
    profileAction.execute(profile);
    profiles.add(profile);
}

Your syntax doesn’t need to change if you inline the profiles in the main build.gradle file, but if you use a fragment, you need to call the profile method with the name of the profile and the configuration closure:

// h2.gradle
profile('h2') {
    ...
}

#3. Remove the wrapper entirely and only include the properties directly on the profile in the fragment.

You end up using the file name as the name for the NamedDomainObjectContainer because it’s all you have at the time you need to create the object:

// plugin
project.fileTree(profileDirectory).matching(files -> files.include("**/*.gradle")).forEach(file -> {
    DatabaseProfile profile = new DatabaseProfile(file.getName().replace(".gradle", ""));
    project.apply(Helper.asMap("from", file, "to", profile));
    dslExtension.getProfiles().add(profile);
});

Not very interesting, but the fragment is literally the ... from the earlier examples. This might make your Gradle fragment look a lot closer to the property file version.

// h2.gradle
...

#4. Use a Groovy class that can intercept the missing method call and properly add it to the NamedDomainObjectContainer,

This gets you closest to the original version, but requires the most amount of code. First create the wrapper that will handle the calls:

class DatabaseProfilesDelegate {

    private final NamedDomainObjectCollection<DatabaseProfile> profiles;

    DatabaseProfilesDelegate(NamedDomainObjectCollection<DatabaseProfile> profiles) {
        this.profiles = profiles;
    }

    void methodMissing(String name, args) {
        DatabaseProfile profile = new DatabaseProfile(name);
        Closure closure = args[0];
        closure.delegate = profile
        closure.call()
        profiles.add(profile);
    }

}

Then you’ll use that class as the target of the to:

// plugin
project.fileTree(profileDirectory).matching(files -> files.include("**/*.gradle")).forEach(file -> {
    project.apply(Helper.asMap("from", file, "to", new DatabaseProfilesDelegate(dslExtension.getProfiles())));
});

With this, the fragment can look exactly the same as if it were in the profiles {} block of the extension:

// h2.gradle
h2 {
    ...
}

Thanks so much again for your help James.

I went with the Groovy #methodMissing approach and it worked like a champ. It was a little more work as you said, but the outcome was exactly what I was looking for. And I was actually able to allow for any of the DSL configs we discussed. E.g. all of these are legal using this approach:

h2.profile
-----------
h2 {
    ...
}

derby.gradle
-----------
profiles {
    derby {
        ...
    }
    derbySpatial {
        ...
    }
}

mysql.gradle
-----------
databases {
    profiles {
        mysql {
            ...
        }
    }
}

:slight_smile: