Source and destination must be different gradle issue after upgrading to Android4

Upgrading the gradle 5.6.5 to 6.1.1 causing this build issue
OS: Windows 10

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ‘:app:externalNativeBuildDebug’.

Source C:xxx\app\build\intermediates\cmake\debug\obj\x86\libxxx.so and destination C:xxx\app\build\intermediates\cmake\debug\obj\x86\libxxx.so must be different

Try doing a clean build, if that doesn’t work then invalidate the caches in Android Studio

I’ve already tried that, thanks.
The problem is that I don’t want to do this each and every time.
I need some stable-permanent solution, or at least understanding why this happens.