Maven hive-jdbc教程

由于项目中需要用到hive-jdbc从数据仓库拉数据下来，所以简单的学一下hive，hive数据仓库建构在hadoop集群之上，数据存在hdfs文件系统中，hive中执行的操作会装换成mapreduce作业进行执行，hive支持类似SQL的语言HQL，hive采用元数据对表进行管理，元数据有三种存放模式：嵌入模式，远程模式，本地模式；hive提供了强大的编程接口，hive jdbc可以让你如使用普通的jdbc一般来操作hive表以及数据。

1.添加依赖

        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-hdfs</artifactId>
            <version>2.6.4</version>
        </dependency>

        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.6.4</version>
        </dependency>

        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-exec</artifactId>
            <version>1.1.0</version>
            <exclusions>
                <exclusion>
                    <artifactId>
                        pentaho-aggdesigner-algorithm
                    </artifactId>
                    <groupId>org.pentaho</groupId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-jdbc</artifactId>
            <version>1.1.0</version>
        </dependency>

2.jdbc连接hive

public class TestHive {
        private static String driverName = "org.apache.hive.jdbc.HiveDriver";//jdbc驱动路径
        private static String url = "jdbc:hive2://hiveserver.xxx.com:10000/dbName";//hive库地址+库名
        private static String user = "username";//用户名
        private static String password = "pwd";//密码
        private static String sql = "";
        private static ResultSet res;

        public static void main(String[] args) {
        Connection conn = null;
        Statement stmt = null;
        try {
            conn = getConn();
            System.out.println(conn);
            stmt = conn.createStatement();
                        String tableName="tab_name";//hive表名
                        sql = "select * from " + tableName;
                System.out.println("Running:" + sql);
                res = stmt.executeQuery(sql);
                System.out.println("执行 select * query 运行结果:");
                while (res.next()) {
                    System.out.println(res.getInt(1) + "\t" + res.getString(2));
                }

        } catch (ClassNotFoundException e) {
            e.printStackTrace();
            System.exit(1);
        } catch (SQLException e) {
            e.printStackTrace();
            System.exit(1);
        } finally {
            try {
                if (conn != null) {
                    conn.close();
                    conn = null;
                }
                if (stmt != null) {
                    stmt.close();
                    stmt = null;
                }
            } catch (SQLException e) {
                e.printStackTrace();
            }
        }
    }

    private static Connection getConn() throws ClassNotFoundException,
            SQLException {
        Class.forName(driverName);
        Connection conn = DriverManager.getConnection(url, user, password);
        return conn;
    }
}

3.查询hive表数据

这个就和普通的jdbc差不太多，也是用sql的方式进行查询，具体的查询语法，可以参考hive官网

4.封装hive工具类

写完了再贴上来

@落雨
ae6623.cn

转载于:https://www.cnblogs.com/ae6623/p/5686504.html