Spark Streaming的maven配置pom.xml

不多说，直接查询相关的内容
https://mvnrepository.com/search?q=spark

我们可以看到很多常见的东西，spark-core,spark-sql,spark-streaming等等
点开这些链接进入到版本列表界面，选择自己需要的配置
spark-core

spark-streaming

这里就不一一说明了，下面我直接列出一个已经测试成功的pom.xml

<properties>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
    <encoding>UTF-8</encoding>
    <scala.version>2.11.12</scala.version>
    <spark.version>2.3.0</spark.version>
    <hadoop.version>2.6.5</hadoop.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
    </dependency>
    
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-common</artifactId>
      <version>${hadoop.version}</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-hdfs</artifactId>
      <version>${hadoop.version}</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-mapreduce-client-common</artifactId>
      <version>${hadoop.version}</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-mapreduce-client-core</artifactId>
      <version>${hadoop.version}</version>
      <scope>provided</scope>
    </dependency>
    
    
    <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core -->
	<dependency>
	    <groupId>org.apache.spark</groupId>
	    <artifactId>spark-core_2.11</artifactId>
	    <version>${spark.version}</version>
	</dependency>
	
	<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming -->
	<dependency>
	    <groupId>org.apache.spark</groupId>
	    <artifactId>spark-streaming_2.11</artifactId>
	    <version>${spark.version}</version>
	    <scope>provided</scope>
	</dependency>
  </dependencies>

这里不建议在dependency中写死version，而是采用一下方式

<hadoop.version>2.6.5</hadoop.version>

在我的配置中有4个hadoop.version，如果写死了，那么后边要进行统一的hadoop升级，你该如何操作？一个一个找，然后修改4个位置？如果这里有上百个依赖，你该如何操作。

这个pom.xml修改完毕，保存后，我的eclipse上爆出了170个problems，但是不要慌，进行如下操作