Spark SQL之jdbc方式访问

首先看thriftserver状态

使用jps查看thriftserver运行状态和端口。如果没启动则需要启动thriftserver

添加依赖

pom.xml文件内容如下：

    <!-- jdbc -->
    <dependency>
      <groupId>org.spark-project.hive</groupId>
      <artifactId>hive-jdbc</artifactId>
      <version>1.2.1.spark2</version>
    </dependency>

编写jdbc代码

package cn.ac.iie.spark

import java.sql.DriverManager

/**
 * 通过JDBC 方式
 */
object SparkSQLThriftServerApp {
  def main(args: Array[String]): Unit = {
    Class.forName("org.apache.hive.jdbc.HiveDriver")
    val conn = DriverManager.getConnection("jdbc:hive2://manager:10000", "iie4bu", "")
    val pstmt = conn.prepareStatement("select transactionid, customerid from sal")
    val rs = pstmt.executeQuery()
    while(rs.next()) {
      println("transactionid:" + rs.getInt("transactionid") + ", customerid:" + rs.getString("customerid"))
    }
    rs.close()
    pstmt.close()
  }

}

输出结果如下：