cdh5.13 spark连接hive数据源

1通过hive直接连接

cdh拷贝的客户端文件缺少相关配置需要修改hive-site.xml 添加

  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://master01:3306/hive?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>username to use against metastore database</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>root</value>
    <description>password to use against metastore database</description>
  </property>

cdh5.13 spark通过hiveserver2连接hive数据源
11引入依赖包

<dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-jdbc</artifactId>
            <version> 1.1.0</version>
 </dependency>
``

def main(args: Array[String]): Unit = {

Class.forName("org.apache.hive.jdbc.HiveDriver")
val conn = DriverManager.getConnection("jdbc:hive2://master01:10000", "root", "")
// val stmt = conn.prepareStatement("select * from tmp.gr_p_customer")
val sql = "show databases"
val rd1= conn.prepareStatement(sql)
val rusult = rd1.execute()
conn.close()

}

注意可能同时通过hiveserver和hive直接相互连接会报一个错误

java.lang.NoSuchFieldError: HIVE_SCRATCH_DIR_LOCK

解决方法：

pom文件中spark-hive以及hive-jdbc包冲突，注释上hive-jdbc即可