我们在配置Spark core 的 maven依赖的时候jackson-module-scala_2.11的版本冲突,解决办法如下:
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.3.1</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
<version>2.9.5</version>
</dependency>
</dependencies>
在Spark core中jackson-module-scala_2.11的版本默认为2.9.3,与下面设置的版本信息有冲突,添加如下信息可以解除冲突:
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.3.1</version>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
<version>2.9.5</version>
</dependency>
</dependencies>
添加信息后确实Maven不会包红了,但是现在在maven中查找的时候jackson-module-scala_2.11没有2.9.3版本的信息,还是会提示没有2.9.3模块,此时我们就指定版本信息为2.9.5而非2.9.3,修改信息如下:
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.3.1</version>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
<version>2.9.5</version>
</dependency>
</dependencies>
</dependencyManagement>
然后刷新Maven,一切就正常了。