Native Binaries: How to share common settings across independent projects

Help/Discuss
1

I’m finding the opportunity for common compiler, linker, visual studio solution generation, and other settings not covered by the default settings for builds in independent projects.

Is there a recommended pattern for sharing such settings across such projects? At the moment, I’m copying this config into each project, but I’m sure this can be achieved another way.

Note

This is not for a monolithic build that manages multiple project builds. This would be for independent projects that happen to reuse the same settings.

2

Should I be looking at custom plugin development? Hearing no suggestions, that’s where I’m going to try starting for now. It’s just not clear to me how I get a plugin to setup default project generation and compiler settings supported by another plugin, but I guess I’ll find out.

3

Are init scripts a viable option? I’m shooting in the dark, here.

4

OK - here’s an apparently dumb attempt to move some common Visual Studio project generation build logic out of a specific project’s build script and into another location.

It doesn’t work, and it’s not clear to me why not. Any guidance is appreciated.

I have a ./buildSrc/build.gradle file with this content:

apply plugin: 'visual-studio'

rootProject.dependencies {
  runtime project(path)
}

rootProject.model {
  visualStudio {
    solutions.all {
        solutionFile.withContent { TextProvider content ->
            content.text = content.text.replace( project.projectDir.getAbsolutePath(), ".")
            content.text = content.text.replaceAll( "Microsoft Visual Studio Solution File, Format Version [0-9]+\\.[0-9]+",
                                                 "Microsoft Visual Studio Solution File, Format Version 12.00")
            content.text = content.text.replace( "# Visual C++ Express 2010", "# Visual Studio 2013" )
            def builder = content.asBuilder()
            def versionIndex = builder.indexOf("# Visual Studio 2013" )
            builder.insert(versionIndex + 20, """
  VisualStudioVersion = 12.0.31101.0
  MinimumVisualStudioVersion = 12.0.21005.1
            """)
        }
    }
    projects.all {
      projectFile.withXml {
        asNode().@ToolsVersion = "12.0"
        asNode().PropertyGroup.findAll({ it.@Label == 'Configuration' }).each { configNode ->
          configNode.appendNode("PlatformToolset", "v120")
        }
      }
    }
  }
}

In my root project, I have only logic relating to compiling the sources for the project in question. If I issue this command in the root project directory:

.\gradlew testComponentVisualStudio

…I get this error:

FAILURE: Build failed with an exception.

* What went wrong:
Task 'testComponentVisualStudio' not found in root project 'cpp.gradle.testComponent'.

So, why am I getting this error? How can I get the visual studio solution/project generation logic to execute for this specific project without having to copy/paste it into the body of the root project’s build script?

5

Gave up on buildSrc/build.gradle.

Now I’m looking into a custom plugin, initially under the buildSrc folder until I get it working.

The goal is for somebody working on a native binary project to apply this custom plugin to their project’s build script, and, with no other configuration or closures specified in their build script, be able to run .\gradlew <component>VisualStudio task and produce the customized Visual Studio solution and project file content defined in this custom plugin.

For example, I’d like for this custom plugin to produce the same results as if I’d had this content in the target project’s build script:

model {
    visualStudio {
      solutions.all {
          solutionFile.withContent { TextProvider content ->
              content.text = content.text.replace( project.projectDir.getAbsolutePath(), ".")
              content.text = content.text.replaceAll( "Microsoft Visual Studio Solution File, Format Version [0-9]+\\.[0-9]+",
                                                   "Microsoft Visual Studio Solution File, Format Version 12.00")
              content.text = content.text.replace( "# Visual C++ Express 2010", "# Visual Studio 2013" )
              def builder = content.asBuilder()
              def versionIndex = builder.indexOf("# Visual Studio 2013" )
              builder.insert(versionIndex + 20, """
    VisualStudioVersion = 12.0.31101.0
    MinimumVisualStudioVersion = 12.0.21005.1
              """)
          }
      }
    }
}

And I’d like for that to be possible by simply applying this custom plugin.

That is, apply plugin: customVisualCppPlugin would be all you’d need to do so that you could run gradlew componentVisualStudio and get this customized Visual Studio solution file out.

So far, all I’ve got is a skeletal plugin that applies the visual-studio plugin like so:

import org.gradle.api.Project;
import org.gradle.api.Plugin;
import org.gradle.ide.visualstudio.plugins.VisualStudioPlugin;

class CommonVisualCppPlugin implements Plugin<Project> {
    void apply(Project theProject) {
        def pluginManager = theProject.getPluginManager();
        pluginManager.apply( VisualStudioPlugin );
    }
}

…but I’m at a complete loss now for how, from this plugin, to apply the common Visual Studio solution file contents. I’m looking for something that will work regardless of whether or not the target project has defined model { visualStudio {} }.

I’ve tried doing simple-seeming operations within the apply() method like:

theProject.extensions.visualStudio { solutions.all { /* custom content here */ } }

…but of course this just flat out fails. I’m pretty much trying all kinds of random stuff that probably has no hope of working. Any suggestions are definitely welcome.

@Daniel_L: sorry to ping you again, but is what I’m trying to achieve even possible? If so, what would it look like? Is a custom plugin even the right approach?

Thanks…

6

There is a lot to discuss about here. The sort answer is yes it’s all possible and everything you tried should work. The long answer is it really depends on the end goal. You told us about what you are trying to achieve which is applying a plugin that configure the visual-studio plugin in your own custom way (related to previous discussion, having a Visual Studio 2013 solution generated plugin). You want to share that configuration between lots of independent project. There is several way to do that. Let’s diverge a bit from the question and explore how to centralized configuration.

  1. build.gradle: Per Gradle project configuration. You can write the code at different level. For example, you can write a general configuration using the Project.allprojects in the root build script.
  2. apply from: This is the extension from the previous, you can isolate the configuration inside a .gradle script that you apply from with a path to the script. This is pretty neat as it enables you to group some boiler plate code inside individual script file and just copy them over a new project to share. The big disadvantage is the versioning of those script. Luckily, the apply from can also use an url for where to find the script. With public git repo served through http, you can apply the raw file directly from the repo (either from master or a tag). Note that any kind of http link will work. This enables you to version them and distribute them across multiple project. The big downside is testing those script can be a pain due to it’s standaloneness.
  3. buildSrc: The first step of a plugin that you can applied. Disclaimer, you can create an plugin inside the build.gradle script but this step is the first real way for having a standalone version of the plugin. This follow the exact same layout as a full plugin project. The big advantage is you don’t have to worry about distribution. The big inconvenient is it can be hard to distribute. Yes advantage and inconvenient. Depending on your need it may be good or bad. It’s also easy to do modification to your plugin code.
  4. plugin: The plugin is a separate project with its own release process. It sounds like an one size fit all solution but be warn that the releasing and consumption of the plugin bring lots of overhead when developing. It is true that your plugin should have test to show the correctness of the code (the same goes for buildSrc) but in reality, for configuration code which is mostly what you need it’s mostly overkill to provide testing code.
  5. contribute a fix: This is sometime a better solution then having a custom piece of code doing lots of configuration when the solution could be better modelled. When you are after fast result, this may not be the best direction, but trust me, in the long run this will become a very good solution that you should have consider at the beginning.

Before I go to a possible solution, you mentioned init scripts. I personally think init script should consider as a last resort and I didn’t find any use case for them yet. If you put the init script in the ~/.gradle folder, it will be applied to every project - ever - ran on your machine. This is a big responsibility as you don’t know what kind of project you may encountered. If you assume all native project and then you start working on a gradle plugin which is Java, you will have problem. If the API change, the script may not work for a specific version of Gradle. If you have some developers on your team not wanting to learn gradle, you will upset them if they have to install the init script. Finally, if you need to update them, how do you version them? Ask the already upset developer to update the init script only to find out they didn’t read your email about updating the init script and now they are having weird failure? You can put the script in a custom distro of Gradle (in the init.d folder) but versioning and maintaining those distro is a pain. The best solution for init script is really using the apply from: where the url points to a raw file on a git repo, for isntance, inside a buildscript dsl. Save you lots of time and headache.

Back to the problem. You seems to be configuring something that should be strait forward. If something is strait forward, the configuration should just be explicit. There is not need to generalize it to much (creating a plugin) because you may later want to upgrade to a new version of Visual Studio only in one project and now you will find yourself that the plugin that you created is not flexible enough so you just ends up copy&pasting the code from your plugin directly in the project. That been said your case is a bit more complex in term of configuration. My gut feeling would be to get some contribution done so this case is handled properly and without exhaustive configuration.

Now, you may be in a time constraint and you “just want it to work” (I can relate). So let’s work on something that will work now for your use case. The buildSrc directory should be seen as a project that has nothing to do with your normal project. Meaning it act totally on its own as a separate plugin which the complex release process was taken out. Having a build.gradle file in there that modify your rootProject just won’t work. What you would want to do, is copy that buildSrc/build.gradle file to, by some convention, gradle/visualStudio2013.gradle. Then in your root build.gradle file, you apply from: ‘gradle/visualStudio2013.gradle’. This act a bit like an #include from C/C++. It will execute the script file where the apply from was called. This way, you share your visualStudio2013.gradle file between your individual project OR you push it at some URL and apply from: . The later is a bit easier to manage updates and follow what version is included inside each project.

If you really want to do a plugin, you will need to use the RuleSource pattern. All legacy plugin (the plugin not using the model) does there work in the apply method. The model introduce a new and better way of doing that. You basically define rules that mutate the model elements. Here are the best videos regarding the model:

In the mutate rule, you would basically put all the code you would normally put under the visualStudio node of the model. The project.extensions.visualStudio {…} is the part of the legacy of Gradle which will be going away when the model is finalized. If no extensions was register by the VisualStudio plugin this won’t work.

Finally, there was a mention about how to apply you logic only if the ‘visual-studio’ plugin is applied. This can be done and it’s very much encouraged to follow this practice (where it make sense) to avoid eagerly applying plugin on project that you don’t want (for example applying the Visual Studio plugin on a Java project). If you use the model or the RuleSource inside a plugin, this is already the behavior as if the node is not instantiated, the rule won’t get applied. In the legacy model, you would use project.pluginManager.withType(VisualStudioPlugin) {…}. This basically register a Closure/Action to be executed when the VisualStudioPlugin or any other plugin inheriting from that plugin is applied.

Long story short, if the configuration becomes complex, there is most-likely a problem with the model and a contribution for a fix would be in order. This takes a lot more time but shouldn’t be discarded as the more configuration code you write the harder it may become using a new version of Gradle and the more version of Gradle you lag behind the harder it gets to move forward.

This was a long answer but I think it covert pretty much everything you asked. Don’t hesitate to ping me in question where I can help you and don’t hesitate to ask more questions.

1 Like
7

Wow - awesome information. Thank you so much for taking the time to detail these options and tradeoffs. I will work through these and revise until I’ve got something workable.

Initially this kind of config will be somewhat fixed but I’m sure I’ll need it to be more flexible and customizable as the range of projects grows.

I hadn’t considered turning it into a contribution, but maybe it’ll end up generally useful enough to do that. I’ll keep that on my list of options.

Thanks again - I’ll post again here when I’ve got it sorted out.

8

@Daniel_L: a first pass at trying out the RuleSource approach is nearly there, but I can’t seem to find a way to get hold of a reference to the project within the @Mutate method. I’d been using the project previously in order to run this substitution in the solution file generation:

solutionFile.withContent { TextProvider content ->
  content.text = content.text.replace( <projectRef>.projectDir.getAbsolutePath(), ".")

Simply adding a Project reference as a second argument to the @Mutate method like so:

    static class Rules extends RuleSource {
      @Mutate
      void applySolutionConfiguration( VisualStudioExtension visualStudio, Project theProject ) {
         visualStudio.solutions.all {
            content.text = content.text.replace( theProject.projectDir.getAbsolutePath(), ".")
         }
      }
   }

…doesn’t work because Project can’t be bound to a model space element, so I get this error:

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring root project 'cpp.gradle.goodbye'.
> The following model rules could not be applied due to unbound inputs and/or subjects:

    VisualCppPlugin.Rules#applySolutionConfiguration
      subject:
        - visualStudio VisualStudioExtension (parameter 1)
      inputs:
        - <no path> Project (parameter 2) [*]

  [*] - indicates that a model item could not be found for the path or type.
9

The answer below to this question:

led me to this approach:

@Mutate
void applySolutionConfiguration( VisualStudioExtension visualStudio,  @Path('buildDir') theBuildDir ) { ... }

…which isn’t the best for the reason highlighted by phaninra above. I’m a little surprised I couldn’t simply do this:

@Mutate
void applySolutionConfiguration( VisualStudioExtension visualStudio,  @Path('projectDir') theProjectDir ) { ... }

…why is buildDir special? Both are properties of the project, I’m surprised all of the File properties of the project aren’t equally accessible.

10

Without going too much into detail, as I’m pretty busy to write something as exhaustive as my previous post, you should look at this class https://github.com/gradle/gradle/blob/master/subprojects/core/src/main/groovy/org/gradle/api/internal/project/ProjectIdentifier.java and it’s usage in https://github.com/gradle/gradle/blob/master/subprojects/ide-native/src/main/groovy/org/gradle/ide/visualstudio/plugins/VisualStudioPlugin.java#L66.

Note the class is internal and has TODO in the class so it may not be a good idea long term but for short term this may be a good solution.

There is other ways to get the projectDir which are hacky and bad but works. The other example that come to my mind is public/protected (depending of the scope) static variable that you set in the apply method and use in your rule. This is very bad and I’m not recommending this. I would use internal api over this.

I hope this help, I will keep an eye for a better solution.