问题描述:
开发环境为spark2.0 + kafka0.8,spark-streaming-kafka-0-8-assembly_2.11.jarsparkStreaming消费kafka时遇到如下错误:
18/02/27 10:19:17 WARN ReceiverSupervisorImpl: Reported error Error handling message; exiting - java.lang.NoSuchMethodError: net.jpountz.util.Utils.checkRange([BII)V
18/02/27 10:19:17 WARN ReceiverTracker: Error reported by receiver for stream 0: Error handling message; exiting - java.lang.NoSuchMethodError: net.jpountz.util.Utils.checkRange([BII)V
at org.apache.kafka.common.message.KafkaLZ4BlockInputStream.read(KafkaLZ4BlockInputStream.java:176)
问题原因:
1.spark2.0用到了lz4-1.3.0.jar,kafka0.8用到了lz4-1.2.0.jar,而程序运行时使用的是lz4-1.3.0.jar。2.lz4-1.3.0.jar包中net.jpountz.util.Utils 类中没有checkRange,该方法位于net.jpountz.util.SafeUtils和net.jpountz.util.Unsa