[Author]: kwu
hbase提供实时查询完好的弥补了hive实时查询的不足,现基于CDH5.4的hive与hbase的整合,已经非常成熟。具体步骤如下:
1、把hive-hbase-handler-1.1.0-cdh5.4.0.jar cp到hbase/lib 下
cp /opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/hive/lib/hive-hbase-handler-1.1.0-cdh5.4.0.jar /opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/hbase2、配置hive-site.xml
<property>
<name>hive.aux.jars.path</name>
<value>file:///opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/hbase/hive-hbase-handler-1.1.0-cdh5.4.0.jar,file:///opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/hbase/lib/hbase.jar,file:///opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/hbase/lib/zookeeper.jar
</value>
</property>
如果在CM中的配置如图:
注意: hive数据路径下,不在保存数据,而在保存在 hbase路径中
insert大量数据时由于WAL比较慢,可以:
set hive.hbase.wal.enabled=false;
3、相关操作命令
创建hive-hbase表
CREATE TABLE cookiekeyhbase(key string ,cookie string)STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "cookiekeyhbase");
插入数据
insert into table cookiekeyhbase select key,cookie from cookiekey limit 150;
全量覆盖导入数据:
insert overwrite table cookiekeyhbase select key,cookie from cookiekey ;
测试结果
1) hive的查询正常
2)hbase中的查询正常