Building a project with Gradle 7.0 --warning-mode all I’m seeing a lot of messages like this (probably one for every source file):
Gradle detected a problem with the following location: ‘/Users/scott/dev/MyNativeProject/build/objs/myNativeProject/shared/debug/MyNativeProject Cpp/7u3bbmnwyd1tohfrj5el5y9wk/SharedMemorySupport.o’. Reason: Task ‘:MyNativeProject:compileMyNativeProjectProductionSharedLibraryMyNativeProjectCpp’ uses this output of task ‘:MyNativeProject:compileMyNativeProjectDebugSharedLibraryMyNativeProjectCpp’ without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to Dealing with validation problems for more details about this problem.
Note the Debug vs Production in the task names.
My project has two buildTypes “debug” and “production” (I didn’t use “release” because of issues I had with the “release” plugin.)
I don’t understand why Gradle thinks that the two buildTypes are depending on each other. They are obviously built from the same source, but the destination folders are unique, and are the defaults that Gradle used (…/shared/debug/… and …/shared/production/… I don’t have anything in the gradle file that I can attribute to that.
Hi Scott, this is indeed a strange error and your original assessment should be correct where each compile task shouldn’t use the other’s output. Without having more context around your build configuration, it may be due to a misconfiguration. I know native build tends to have a more complicated configuration scheme which can be easy to mix up. If you could reproduce the issue in a smaller build, we would look to see what’s happening.
It gets weirder… I took the project and started to strip it down to create a reproducible example. When I eliminated certain source files - that is just some .cpp and .h files - keeping the .gradle files the same, the problem disappeared. I’m wondering if somewhere in the included header files there are #pragma directives that are messing it up? The code that is triggering this is using the Boost library (v1.66.0) to work with shared memory. I’m going to see if I can narrow it down, but I don’t have a lot of time to spend on it.
Thanks for looking for a reproducer. It’s indeed quite strange. However, given you are using the Boost library, there is a chance the fallback strategy is used when walking the headers. There are side effects especially if the headers are located in the project directory directly. What’s the source layout of your project look like? Is it following the default layout, that is src/main/cpp/src/main/headers or something else?
The project structure was adapted from an existing Visual Studio project a long time ago. On Windows we are not using the Gradle plugins to build, instead calling our own custom task to run the original Visual Studio build. The failure I’m observing is on macOS, I haven’t tried Linux yet.
The project structure does not follow the defaults. C/C++ source files are in the same folder as the build.gradle file
We are extracting the Boost SDK into a folder that is outside the project hierarchy and defining a pre-built library for it as well as headers for the Java Native Interface. That part of the Gradle file looks like this:
model {
repositories {
println "Defining pre-built libraries for JVM and generated headers..."
libs(PrebuiltLibraries) { libs ->
// Add the JVM includes
jvm {
headers.srcDir "${JAVA_HOME}/include"
binaries.all {
def os = targetPlatform.operatingSystem
if (os.windows) {
headers.srcDir "${JAVA_HOME}/include/win32"
} else if (os.macOsX) {
headers.srcDir "${JAVA_HOME}/include/darwin"
} else {
headers.srcDir "${JAVA_HOME}/include/linux"
}
}
}
boost {
headers.srcDir "${INTERNAL_COMPANY_SDK}/ExternalSDKs/company.domain.thirdparty/boost-${boostVersion}-sdk/boost_${boostVersion.replace(".","_")}"
}
}
}
}
The main part of the build file looks like this:
model {
platforms {
arm {
architecture "arm"
}
aarch64 {
architecture "arm64"
}
x86 {
architecture "x86"
}
x64 {
architecture "x86_64"
}
}
buildTypes {
debug
// The net.researchgate.release plugin destroys any 'release' build
// type. That in turn causing task names for compiling to change.
//release
production // so we use 'production' as a synonym to 'release'
}
//
// Toolchain for cross-compiling must be installed
// sudo apt-get install gcc-arm-linux-gnueabihf
//
// We make a folder with standard executable names linked to the arm versions
// sudo mkdir /opt/arm/bin
// sudo ln -s /usr/bin/arm-linux-gnueabihf-gcc /opt/arm/bin/gcc
// sudo ln -s /usr/bin/arm-linux-gnueabihf-g++ /opt/arm/bin/g++
toolChains {
gcc(Gcc) {
target('x64') {
compilerProbeArgs '-m64'
}
target('x86') {
compilerProbeArgs '-m32', '-march=i686'
}
}
clang(Clang) {
target('x64') {
compilerProbeArgs '-m64'
}
}
gccArm(Gcc) {
target('arm') {
path "${ARM_GCC_TOOLS_DIR}/bin" // this folder must be created manually
// TI embedded board is 32-bit
cppCompiler.withArguments { args ->
args << "-I${ARM_GCC_TOOLS_DIR}/include"
}
}
}
gccArm64(Gcc) {
target('aarch64') {
path "${ARM64_GCC_TOOLS_DIR}/bin" // this folder must be created manually
// nVidia Jetson embedded board is 64-bit
cppCompiler.withArguments { args ->
args << "-I${ARM64_GCC_TOOLS_DIR}/include"
}
}
}
}
components {
GradleNativeTest(NativeLibrarySpec) {
// only build shared libraries
binaries.withType(StaticLibraryBinarySpec) { buildable = false }
targetPlatform 'x64'
if(platform != 'mac') {
targetPlatform 'x86'
}
if (platform == 'linux') {
targetPlatform 'arm'
targetPlatform 'aarch64'
}
sources {
cpp {
// Always depend on the JVM JNI headers
lib library: 'jvm', linkage: 'api'
lib library: 'boost', linkage: 'api'
preCompiledHeader "stdafx.h"
source {
srcDirs '.'
include '*.cpp'
if (!System.getProperty('os.name').startsWith('Windows')) {
exclude 'dllmain.cpp'
exclude 'stdafx.cpp'
}
}
exportedHeaders {
srcDirs '.'
}
}
}
}
}
}
The Boost SDK is in a zip file in our artifact repo and is extracted as part of the build:
I see the issue…sight. I really want to fix that issue directly, but it’s pretty deep in the code so it will take me some time in Nokee to work through the Gradle code to provide a fix. The problem only happens in a very precise scenario. When the header search directory is in a project directory (exportedHeaders.srcDirs '.') and the header parsing encounter a case where it can’t figure out the complete header graph, i.e. a macro include (#include MACRO), typical in Boost headers, the compile tasks snapshots all files found in the header search directories. In general, it’s not much of an issue, but for this particular case, the build directory is also snapshotted Basically the compile tasks ends up “using” the output of every tasks. Gradle 7 is more strict so you are seeing the warning. It is a very good warning to get as most likely your build must have had some issue being fully up-to-date.
That being said, it’s a shame this is still an issue and would love for 2021 to be the last year this is an issue… The best all-around solution is to isolate the headers prior to building. It’s a minimal change to your build, doesn’t require moving any code around and nicely solve the issue. In theory, you should be able to use the same solution developed for the Nokee plugins.
Note that you may have to do is return the base directory of the FileTree instead of the FileTree itself. However, you would need to return the base directory as a Task producer aware type, e.g. Provider. The reason for the note is the SourceDirectorySet will only accept directories and a FileTree includes files from a base directory. Nokee has some added smartness to work around this issue naturally. As the Sync task is referenced as a TaskProvider, you can easily map the destinationDir: syncTask.map { it.destinationDir } This will create a provider of the isolated header search path directory with the producer task knowledge.
I hope that makes sense, don’t hesitate to ask any question on any part that I may have misexplained. It’s unfortunately not an easy problem to fix.
Basically the compile tasks ends up “using” the output of every tasks. Gradle 7 is more strict so you are seeing the warning. It is a very good warning to get as most likely your build must have had some issue being fully up-to-date.