因为jar冲突,es client使用的httpjar被忽略,所以导致报错,所以使用以下命令确保用户的jar被优先加载。
spark-submit --class SparkTest --master yarn --deploy-mode client \
--driver-cores 1 \--driver-memory 1G \
--num-executors 1 \
--executor-cores 1 \
--executor-memory 1G \
--conf spark.io.compression.codec=org.apache.spark.io.LZ4CompressionCodec \
--conf spark.serializer=org.apache.spark.serializer.JavaSerializer \
--conf spark.executor.userClassPathFirst=true \
es-1.0-SNAPSHOT.jar