Gradle build fail for avro-mapred hadoop2 jar

I have a build.gradle file as follows :

plugins {
    id 'java'
}

group 'org.example'
version '1.0-SNAPSHOT'

repositories {
    mavenCentral()
}



dependencies {
    implementation group: 'org.apache.avro', name: 'avro-tools', version:'1.10.2'
    implementation group: 'org.apache.spark', name: 'spark-core_2.12', version:'2.4.1'
}

test {
    useJUnitPlatform()
}

I get the following error out put :

Could not find avro-mapred-1.10.2-hadoop2.jar (org.apache.avro:avro-mapred:1.10.2).
Searched in the following locations:
    https://repo.maven.apache.org/maven2/org/apache/avro/avro-mapred/1.10.2/avro-mapred-1.10.2-hadoop2.jar

Possible solution:
 - Declare repository providing the artifact, see the documentation at https://docs.gradle.org/current/userguide/declaring_repositories.html

Your spark-core_2.12 dependency has a dependency on org.apache.avro:avro-mapred:1.8.2:hadoop2.
Your avro-tools dependency has a dependency on org.apache.avro:avro-mapred:1.10.2.
So 1.10.2 wins, but in 1.10.2 there is no artifact with hadoop2 classifier anymore.
So without knowing spark or avro I’d say that spark 2.4.1 is simply not compatible with avro 1.10.2.

1 Like