使用spark操作hive库的数据,在日常业务中随处可见,记录一下……
话不多说
1、无论使用idea或者eclipse,创建工程后,spark、hive 的相关pom
.
<properties>
<spark.core.version>2.4.0</spark.core.version>
<scala.version>2.11</scala.version>
<build_scope>compile</build_scope>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.version}</artifactId>
<version>${spark.core.version}</version>
<scope>${build_scope}</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_${scala.version}</artifactId>
<version>${spark.core.version}</version>
<scope>${build_scope}</scope>
</dependency>
<dependency>