运行SparkStreaming 示例代码报错：Caused by: java.lang.NoSuchMethodError: net.jpountz.lz4.LZ4BlockInputStream

1、问题描述：

运行官网文档中Spark Streaming 读取socket流 实例代码报错：

环境为kafka 0.9 + Spark 2.4

Caused by: java.lang.NoSuchMethodError: net.jpountz.lz4.LZ4BlockInputStream.<init>(Ljava/io/InputStream;Z)V
	at org.apache.spark.io.LZ4CompressionCodec.compressedInputStream(CompressionCodec.scala:122)
	at org.apache.spark.serializer.SerializerManager.wrapForCompression(SerializerManager.scala:163)
	at org.apache.spark.serializer.SerializerManager.wrapStream(SerializerManager.scala:124)
	at org.apache.spark.shuffle.BlockStoreShuffleReader$$anonfun$3.apply(BlockStoreShuffleReader.scala:50)
	at org.apache.spark.shuffle.BlockStoreShuffleReader$$anonfun$3.apply(BlockStoreShuffleReader.scala:50)

2、原因

net.jpountz.lz4 主要是这个依赖冲突导致的

kafka 0.9中是1.2版本的，而Spark中是1.3版本的

3、解决

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.11</artifactId>
            <version>${kafka.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>net.jpountz.lz4</groupId>
                    <artifactId>lz4</artifactId>
                </exclusion>
            </exclusions>
        </dependency>