How do I link a C++ library with another library file that I know the path of

swpalmer · June 9, 2014, 2:47pm

I have a set of libraries and headers in a known location. I can easily add the headers to the include search path with, for example:

task.withType(CppCompile) {

includes.from("$JAVA_HOME/include")

includes.from("$JAVA_HOME/include/win32") }

But I also need to link with, for example “$JAVA_HOME/lib/jawt.lib”

I’ve used the JDK in my example, but in my real application I will have set of arbitrary folders where I know the path to the .lib or .so files and the includes. The libraries and include paths that I need to add are computed at build time from other metadata that is part of the project (e.g. an XML file that also ships with the product).

daz · July 4, 2014, 5:22pm

Sorry for the delayed reply.

Take a look at the ‘prebuilt-libraries’ sample in the complete distribution for an example of how to declare a dependency on an existing library with known paths.

swpalmer · July 5, 2014, 2:08pm

Thanks, that’s a start. I do have a requirement to programmatically find and name these libraries though. The example has explicitly defined two libraries ‘boost’ and ‘util’ and so it has:

repositories {
        libs(PrebuiltLibraries) {
            boost {
                headers.srcDir "3rd-party-lib/boost_1_55_0/boost"
            }
            util {
                headers.srcDir "3rd-party-lib/util/src/util/headers"
                binaries.withType(StaticLibraryBinary) {
...

What is the syntax if the names of the libraries are discovered at build time by reading another file or scanning a folder?

E.g. imagine I have an XML file like so:

<dependency>xyz.module</dependency>
    <dependency>abc.module</dependency>

and I know that the headers and binaries of those moduels will be located at a specific path, e.g. “libraries/xyz/headers”, “libraries/xzy/win32”, “libraries/xyz/OSX” , etc.

I can see how to set the location of the libraries based on the prebuilt sample, but I don’t know the syntax for declaring a new library dependency that is discovered while running the build script.

daz · July 5, 2014, 7:00pm

We haven’t put a lot of work into refining this mechanism, but most things are possible since the Gradle build file is actually Groovy code.

You should be able to programmatically create PrebuiltLibrary instances based on some configuration with something like (untested):

repositories {
    libs(PrebuiltLibraries) { libs ->
        def myLibraryList = ['xyz', 'abc'] // dummy list
          myLIbraryList.each { myLibrary ->
            libs.create(myLibrary) {
                headers.srcDir "libraries/${myLibrary}/headers"
                .... more library config
            }
        }
    }
}

This integration test has an example of how to programmatically set the paths of a PrebuiltLibrary: https://github.com/gradle/gradle/blob/master/subprojects/cpp/src/integTest/groovy/org/gradle/nativebinaries/language/cpp/PrebuiltLibrariesIntegrationTest.groovy#L83

There’s a lot we could do (and plan to do) to make this easier to configure. We’re actively working on true variant-aware dependency management, which should lead to improvements in this area.

swpalmer · January 15, 2015, 2:49pm

Thanks for your answer. I was able to make a lot of progress, but now I have a new requirement. The native code that I am building will have both 32 and 64-bit versions. The pre-built libraries that it will depend on will also have both 32 and 64-bit versions of the binary to link to. So I need to have the 32-bit output of my build link with the 32-bit version of the pre-built libraries and the 64-bit binary of my build link with the 64-bit version of the pre-built library.

I have this in my project (it’s added via a custom plugin):

project.model {
 buildTypes {
  debug
  release
 }
 platforms {
  windows_x86 {
   architecture 'i386'
   operatingSystem 'windows'
  }
  windows_x64 {
   architecture 'x64'
   operatingSystem 'windows'
  }
  os_x {
   architecture 'x86_64'
   operatingSystem 'mac os x'
  }
  linux_x86 {
   architecture 'i386'
   operatingSystem 'linux'
  }
  linux_amd64 {
   architecture 'amd64'
   operatingSystem 'linux'
  }
 }
}

So the include files for a library will be in one location, say: myLibraryDir/include The binaries to link with will be in: myLibraryDir/bin/Win32 or myLibraryDir/bin/x64 depending on the platform: windows_x86 or windows_x64

It just so happens that the output of my build will be a library that is exactly like the pre-built dependencies. So I will build both the 32 and 64-bit platforms for the host OS in one go.

My current code that adds the dependency on the pre-built libraries looks like this:

project.model {
 repositories {
  libs(PrebuiltLibraries) {
   myLibraryDependencies.each { dep ->
    println "Creating pre-built library dependency: ${dep.libName} with linkage ${dep.linkage} and binary ${dep.binary}"
    libs.create(dep.libName) {
     headers.srcDir dep.headerDir
     binaries.withType(StaticLibraryBinary) {
      if ('static'.equals(dep.linkage)) {
       staticLibraryFile = dep.binary
      }
     }
     binaries.withType(SharedLibraryBinary) {
      if ('shared'.equals(dep.linkage)) {
       sharedLibraryFile = dep.binary
      }
     }
    }
   }
  }
 }
}

As you can see there is nothing differentiating the platform architecture yet. Do I need to treat 32 and 64-bit versions of the pre-built library as different libraries, or can a library have multiple architectures (platforms) defined so the C++ plugin will link to the matching platform automatically? If that isn’t how it works, how do I add a library to only a specific platform? (So I can have win32 version of the library that only the windows_x86 platform depends on and a x64 version of the library that only the windows_x64 platform depends on.)

daz · January 15, 2015, 7:24pm

You need to configure your PrebuiltLibrary so that different variants have different files. We do something similar to this (for operating system) in the ‘native-binaries/prebuilt’ sample.

repositories {
  libs(PrebuiltLibraries) {
    mylib {
      binaries.withType(StaticLibraryBinary) { binary ->
        // This closure will be executed once for every buildType/platform combination
        def variantDir = binary.targetPlatform.architecture.name == 'i386' ? 'Win32' : 'x64'
        staticLibraryFile = "path/to/library/${variantDir}/mylib.so"
      }
    }
  }
}

Of course, you might want to change you code so that you can use “${dep.baseDir}/${variantDir}/${dep.binaryName}.”

swpalmer · February 17, 2015, 4:11pm

What’s the syntax for doing this with Gradle 2.3?

I’m trying to migrate my plugin to the new format and I’m getting errors such as:

Cannot create a ArtifactRepository named ‘binary’ because this container does not support creating elements by name alone. Please specify which subtype of ArtifactRepository to create. Known subtypes are: PrebuiltLibraries

Marco_Erdmann · February 17, 2015, 5:21pm

If you don’t mind, I can give you a full example. I had to figure out most of the stuff a while ago and just ported the whole thing to Gradle 2.3 with the help of Daz. The build.gradle is quite compilicated and I will just give you the native part if you don’t mind, but it should get you going:

apply plugin: 'cpp'
  // C++ configuration
model {
  components {
    libqrblastrs(NativeLibrarySpec) {
              // specify target platforms, necessary for
2.3
      targetPlatform "x64"
      targetPlatform "x86"
        sources {
        cpp {
          def additionalBoostLinkage = System.getProperty("os.name") == "Mac OS X" ? 'shared' : 'api'
          lib library: 'oci', linkage: 'shared'
          lib library: 'cpprest', linkage: 'shared'
          lib library: 'easylogging', linkage: 'api'
          lib library: 'boost', linkage: 'api'
          lib library: 'boost_chrono', linkage: additionalBoostLinkage
          lib library: 'boost_thread', linkage: additionalBoostLinkage
          lib library: 'boost_system', linkage: additionalBoostLinkage
        }
      }
          baseName 'qrblastrs'
        binaries.withType(SharedLibraryBinarySpec) {
        if (toolChain in VisualCpp) {
          cppCompiler.define 'LIBQRBLASTRS_EXPORTS'
          cppCompiler.define 'WIN32'
          cppCompiler.define '_WINDOWS'
          cppCompiler.define '_USRDLL'
          cppCompiler.define '_WINDLL'
          cppCompiler.define '_UNICODE'
          cppCompiler.define 'UNICODE'
          cppCompiler.define '_EXPORTING'
            if (buildType == buildTypes.debug) {
            cppCompiler.define '_DEBUG'
            cppCompiler.args '/Zi', '/nologo', '/W3', '/WX-', '/sdl', '/Od', '/Gm', '/EHsc', '/RTC1', '/MDd', '/Gs', '/fp:precise', '/Zc:wchar_t', '/Zc:forScope', '/Gd', '/TP'
            if (targetPlatform == platforms.x64) {
              linker.args '/INCREMENTAL', '/NOLOGO', '/MANIFEST', '/manifest:embed', '/DEBUG', '/SUBSYSTEM:WINDOWS', '/TLBID:1', '/DYNAMICBASE', '/NXCOMPAT', '/MACHINE:X64', '/DLL'
            } else {
              linker.args
'/INCREMENTAL', '/NOLOGO', '/MANIFEST', '/manifest:embed', '/DEBUG', '/SUBSYSTEM:WINDOWS', '/TLBID:1', '/DYNAMICBASE', '/NXCOMPAT', '/MACHINE:X86', '/DLL'
            }
          } else {
            cppCompiler.define 'NDEBUG'
            cppCompiler.args '/Zi', '/nologo', '/W3', '/WX-', '/sdl', '/O2', '/Oi', '/GL', '/Gm-', '/EHsc', '/MD', '/GS', '/Gy', '/fp:precise', '/Zc:wchar_t', '/Zc:forScope', '/Gd', '/TP'
            if (targetPlatform == platforms.x64) {
              linker.args '/INCREMENTAL:NO', '/NOLOGO', '/MANIFEST', '/manifest:embed', '/DEBUG', '/SUBSYSTEM:WINDOWS', '/OPT:REF', '/OPT:ICF', '/TLBID:1', '/DYNAMICBASE', '/NXCOMPAT', '/MACHINE:X64', '/DLL', '/LTCG'
            } else {
              linker.args '/INCREMENTAL:NO', '/NOLOGO', '/MANIFEST', '/manifest:embed', '/DEBUG', '/SUBSYSTEM:WINDOWS', '/OPT:REF', '/OPT:ICF', '/TLBID:1', '/DYNAMICBASE', '/NXCOMPAT', '/MACHINE:X86', '/DLL', '/LTCG'
            }
          }
        }
        if (toolChain in Gcc) {
          buildable = !(targetPlatform == platforms.x86 && targetPlatform.operatingSystem.linux)
            if (buildType == buildTypes.debug) {
            cppCompiler.args '-g', '-O0', '-std=c++11', '-fPIC'
          } else {
            cppCompiler.args '-O3', '-march=core2', '-std=c++11', '-fPIC'
          }
        }
        if (toolChain in Clang) {
          buildable = !(targetPlatform == platforms.x86 && targetPlatform.operatingSystem.macOsX)
            if (buildType == buildTypes.debug) {
            cppCompiler.args '-g', '-O0', '-std=c++11', '-fPIC'
          } else {
            cppCompiler.args '-O3', '-march=core2', '-std=c++11', '-fPIC'
          }
        }
      }
      binaries.withType(StaticLibraryBinarySpec) { buildable = false }
    }
      libqrblastrsjni(NativeLibrarySpec) {
              // specify target platforms, necessary for 2.3
      targetPlatform "x64"
      targetPlatform "x86"
              sources {
        cpp {
          lib library: 'libqrblastrs', linkage: 'shared'
          lib library: 'jvm', linkage: 'api'
          lib library: 'easylogging', linkage: 'api'
        }
      }
        baseName 'qrblastrsjni'
        binaries.withType(SharedLibraryBinarySpec) {
        if (toolChain in VisualCpp) {
          cppCompiler.define 'LIBQRBLASTRSJNI_EXPORTS'
          cppCompiler.define 'WIN32'
          cppCompiler.define '_WINDOWS'
          cppCompiler.define '_USRDLL'
          cppCompiler.define '_WINDLL'
          cppCompiler.define '_UNICODE'
          cppCompiler.define 'UNICODE'
          cppCompiler.define '_EXPORTING'
          if (buildType == buildTypes.debug) {
            cppCompiler.define '_DEBUG'
            cppCompiler.args '/Zi', '/nologo', '/W3', '/WX-', '/sdl', '/Od', '/Gm', '/EHsc', '/RTC1', '/MDd', '/Gs', '/fp:precise', '/Zc:wchar_t', '/Zc:forScope', '/Gd', '/TP'
            if (targetPlatform == platforms.x64) {
              linker.args '/INCREMENTAL', '/NOLOGO', '/MANIFEST', '/manifest:embed', '/DEBUG', '/SUBSYSTEM:WINDOWS', '/TLBID:1', '/DYNAMICBASE', '/NXCOMPAT', '/MACHINE:X64', '/DLL'
            } else {
              linker.args '/INCREMENTAL', '/NOLOGO', '/MANIFEST', '/manifest:embed', '/DEBUG', '/SUBSYSTEM:WINDOWS', '/TLBID:1', '/DYNAMICBASE', '/NXCOMPAT', '/MACHINE:X86', '/DLL'
            }
          } else {
            cppCompiler.define 'NDEBUG'
            cppCompiler.args '/Zi', '/nologo', '/W3', '/WX-', '/sdl', '/O2', '/Oi', '/GL', '/Gm-', '/EHsc', '/MD', '/GS', '/Gy', '/fp:precise', '/Zc:wchar_t', '/Zc:forScope', '/Gd', '/TP'
            if (targetPlatform == platforms.x64) {
              linker.args '/INCREMENTAL:NO', '/NOLOGO', '/MANIFEST', '/manifest:embed', '/DEBUG', '/SUBSYSTEM:WINDOWS', '/OPT:REF', '/OPT:ICF', '/TLBID:1', '/DYNAMICBASE', '/NXCOMPAT', '/MACHINE:X64', '/DLL', '/LTCG'
            } else {
              linker.args '/INCREMENTAL:NO', '/NOLOGO', '/MANIFEST', '/manifest:embed', '/DEBUG', '/SUBSYSTEM:WINDOWS', '/OPT:REF', '/OPT:ICF', '/TLBID:1', '/DYNAMICBASE', '/NXCOMPAT', '/MACHINE:X86', '/DLL', '/LTCG'
            }
          }
        }
        if (toolChain in Gcc) {
          buildable = !(targetPlatform == platforms.x86 && targetPlatform.operatingSystem.linux)
            if (buildType == buildTypes.debug) {
            cppCompiler.args '-g', '-O0', '-std=c++11', '-fPIC'
          } else {
            cppCompiler.args '-O3', '-march=core2', '-std=c++11', '-fPIC'
          }
        }
        if (toolChain in Clang) {
          buildable = !(targetPlatform == platforms.x86 && targetPlatform.operatingSystem.macOsX)
            if (buildType == buildTypes.debug) {
            cppCompiler.args '-g', '-O0', '-std=c++11', '-fPIC'
          } else {
            cppCompiler.args '-O3', '-march=core2', '-std=c++11', '-fPIC'
          }
        }
      }
      binaries.withType(StaticLibraryBinarySpec) { buildable = false }
    }
  }
    buildTypes {
    debug
    release
  }
    platforms {
    x86 { architecture "x86" }
    x64 { architecture "x86_64" }
  }
    repositories {
    libs(PrebuiltLibraries) {
      oci {
        headers.srcDir "contrib/oracle/include"
        binaries.withType(SharedLibraryBinary) {
          if (targetPlatform.operatingSystem.windows) {
            if (targetPlatform == platforms.x64) {
              sharedLibraryFile = file("file://c/program files/quattro research/OracleInstantClient/oci.dll")
              sharedLibraryLinkFile = file("contrib/oracle/lib/x64/oci.lib")
            } else {
              sharedLibraryFile = file("file://c/program files (x86)/quattro research/OracleInstantClient/oci.dll")
              sharedLibraryLinkFile = file("contrib/oracle/lib/Win32/oci.lib")
            }
          } else {
            def oracleHome = System.getenv("ORACLE_HOME")
            if (file("${oracleHome}/lib/libclntsh.so").exists()) {
              sharedLibraryFile = file("${oracleHome}/lib/libclntsh.so")
            } else if (file("${oracleHome}/libclntsh.so").exists()) {
              sharedLibraryFile = file("${oracleHome}/libclntsh.so")
            } else {
              sharedLibraryFile = file("${oracleHome}/libclntsh.dylib")
            }
          }
        }
      }
        cpprest {
        headers.srcDir "contrib/cpprest/include"
        binaries.withType(SharedLibraryBinary) {
          if (targetPlatform.operatingSystem.windows) {
            if (targetPlatform == platforms.x64) {
              if (buildType == buildTypes.debug) {
                sharedLibraryFile = file("packages/cpprestsdk.2.4.0.1/build/native/bin/x64/v120/Debug/Desktop/cpprest120d_2_4.dll")
                sharedLibraryLinkFile = file("packages/cpprestsdk.2.4.0.1/build/native/lib/x64/v120/Debug/Desktop/cpprest120d_2_4.lib")
              } else {
                sharedLibraryFile = file("packages/cpprestsdk.2.4.0.1/build/native/bin/x64/v120/Release/Desktop/cpprest120_2_4.dll")
                sharedLibraryLinkFile = file("packages/cpprestsdk.2.4.0.1/build/native/lib/x64/v120/Release/Desktop/cpprest120_2_4.lib")
              }
            } else {
              if (buildType == buildTypes.debug) {
                sharedLibraryFile = file("packages/cpprestsdk.2.4.0.1/build/native/bin/Win32/v120/Debug/Desktop/cpprest120d_2_4.dll")
                sharedLibraryLinkFile = file("packages/cpprestsdk.2.4.0.1/build/native/lib/Win32/v120/Debug/Desktop/cpprest120d_2_4.lib")
              } else {
                sharedLibraryFile = file("packages/cpprestsdk.2.4.0.1/build/native/bin/Win32/v120/Release/Desktop/cpprest120_2_4.dll")
                sharedLibraryLinkFile = file("packages/cpprestsdk.2.4.0.1/build/native/lib/Win32/v120/Release/Desktop/cpprest120_2_4.lib")
              }
            }
          } else {
            if (file("/usr/lib64/libcpprest.so").exists()) {
              sharedLibraryFile = file("/usr/lib64/libcpprest.so")
            } else if (file("/usr/lib/libcpprest.so").exists()) {
              sharedLibraryFile = file("/usr/lib/libcpprest.so")
            } else {
              sharedLibraryFile = file("/usr/local/lib/libcpprest.dylib")
            }
          }
        }
      }
        easylogging { headers.srcDir "contrib/easylogging++/include" }
        jvm {
        def javaHome = System.getenv("JAVA_HOME")
        headers.srcDir "${javaHome}/include"
        binaries.all {
          if (targetPlatform.operatingSystem.windows) {
            headers.srcDir "${javaHome}/include/win32"
          } else if (targetPlatform.operatingSystem.linux) {
            headers.srcDir "${javaHome}/include/linux"
          } else {
            headers.srcDir "${javaHome}/include/darwin"
          }
        }
      }
        // additional libraries on Linux && OS X
      boost {
        def boostRoot = System.getenv("BOOST_ROOT")
        if (boostRoot != "") {
          headers.srcDir "${boostRoot}/include"
        }
      }
        boost_chrono {
        def boostRoot = System.getenv("BOOST_ROOT")
        if (boostRoot != "") {
          binaries.withType(SharedLibraryBinary) {
            if (targetPlatform.operatingSystem.macOsX) {
              sharedLibraryFile = file("${boostRoot}/lib/libboost_chrono.dylib")
            }
          }
        }
      }
        boost_thread {
        def boostRoot = System.getenv("BOOST_ROOT")
        if (boostRoot != "") {
          binaries.withType(SharedLibraryBinary) {
            if (targetPlatform.operatingSystem.macOsX) {
              sharedLibraryFile = file("${boostRoot}/lib/libboost_thread-mt.dylib")
            }
          }
        }
      }
        boost_system {
        def boostRoot = System.getenv("BOOST_ROOT")
        if (boostRoot != "") {
          binaries.withType(SharedLibraryBinary) {
            if (targetPlatform.operatingSystem.macOsX) {
              sharedLibraryFile = file("${boostRoot}/lib/libboost_system.dylib")
            }
          }
        }
      }
    }
  }
}

The script generates two native libraries where one is a JNI library linking against the first library. Different pre-built libraries have to be linked depending on the Platform the build is running on. Only Windows is generatiing x86 and x86_64 builds while OS X and Linux compile only for 64-bit. Feel free to ask if you find anything fishy. I’m no expert and went through quite a lot of trial and error, but the build works on all 3 target platform.

swpalmer · February 19, 2015, 2:41pm

Thanks for the response. I have managed to get most of my code working.

I still struggle to understand the gradle model though.

What I had failed to grasp was the context relevant to “binaries.withType(…)”. I had always thought that it was referring to the binaries I had defined and that would be produced by the build, but it apparently depends on the context and can refer to the binaries of a shared or static library that I am defining. Yet somehow that set is magically built from the target platforms of the binaries that I am producing… and the PrebuiltPLibraries, of course don’t actually have any binaries until I assign files to link with… It makes my head spin I’m just beginning to grasp what Gradle’s model is like and how to manipulate it. (I hope!)

If anyone has some pointers on what I must read to properly understand this stuff (other than Gradle source code), please let me know.

Topic		Replies	Views
How do I specify different include paths for a PrebuiltLibrary on a Native C++ application? Help/Discuss roadmap , plugins	1	4274	July 24, 2015
How do you link a native shared library with a relative path? Help/Discuss	3	1720	November 14, 2017
How to specify boost include path? Help/Discuss	1	368	August 14, 2022
Gradle 3.0 - PrebuiltLibraries dependency link error Help/Discuss	2	1199	November 27, 2017
Finding a library for native software Help/Discuss	1	778	October 8, 2018

How do I link a C++ library with another library file that I know the path of

Related Topics