Your spark-core_2.12 dependency has a dependency on org.apache.avro:avro-mapred:1.8.2:hadoop2.
Your avro-tools dependency has a dependency on org.apache.avro:avro-mapred:1.10.2.
So 1.10.2 wins, but in 1.10.2 there is no artifact with hadoop2 classifier anymore.
So without knowing spark or avro I’d say that spark 2.4.1 is simply not compatible with avro 1.10.2.
1 Like