Error publishing build scan - Connection refused

Info from the build:

Gradle version: 6.4
Plugin version: 3.3.1
Request URL: https://scans-in.gradle.com/scans/publish/gradle/3.3.1/token
Request ID: b3d7ca8c-8376-4ae9-ad24-bd96c87f3935
Exception: java.net.ConnectException: Connection refused (Connection refused)

After running simple ./gradlew --scan i see:

Publishing build scan...
Publishing build scan failed due to network error 'java.net.ConnectException: Connection refused (Connection refused)' (2 retries remaining)...
Publishing build scan failed due to network error 'java.net.ConnectException: Connection refused (Connection refused)' (1 retry remaining)...

A network error occurred.

If you require assistance with this problem, please report it via https://gradle.com/help/plugin and include the following information via copy/paste.

I finally managed to get the scan working when I manually override enterprise plugin version back to 3.1.1:

plugins {
    id("com.gradle.enterprise") version "3.1.1"
}

Any newer version seems to result in this error. Please let me know if I can provide any additional info.