HBase与Hive的集成在最新的两个版本中无法兼容。所以,我们只能含着泪勇敢的重新编译:hive-hbase-handler-2.3.7.jar!!
环境准备
因为我们后续可能会在操作Hive的同时对HBase也会产生影响,所以Hive需要持有操作HBase的Jar,那么接下来拷贝Hive所依赖的Jar包(或者使用软连接的形式)
。
告诉hive,hbase放到哪了?
vim hive-env.sh
export HBASE_HOME=/opt/hbase-2.2.2
export HBASE_HOME=/opt/hbase-2.2.2
export HIVE_HOME=/opt/hive-2.3.7
ln -s $HBASE_HOME/lib/hbase-common-2.2.2.jar $HIVE_HOME/lib/hbase-common-2.2.2.jar
ln -s $HBASE_HOME/lib/hbase-server-2.2.2.jar $HIVE_HOME/lib/hbase-server-2.2.2.jar
ln -s $HBASE_HOME/lib/hbase-client-2.2.2.jar $HIVE_HOME/lib/hbase-client-2.2.2.jar
ln -s $HBASE_HOME/lib/hbase-protocol-2.2.2.jar $HIVE_HOME/lib/hbase-protocol-2.2.2.jar
ln -s $HBASE_HOME/lib/hbase-it-2.2.2.jar $HIVE_HOME/lib/hbase-it-2.2.2.jar
ln -s $HBASE_HOME/lib/htrace-core-3.1.0-incubating.jar $HIVE_HOME/lib/htrace-core-3.1.0-incubating.jar
ln -s $HBASE_HOME/lib/hbase-hadoop2-compat-2.2.2.jar $HIVE_HOME/lib/hbase-hadoop2-compat-2.2.2.jar
ln -s $HBASE_HOME/lib/hbase-hadoop-compat-2.2.2.jar $HIVE_HOME/lib/hbase-hadoop-compat-2.2.2.jar
同时在hive-site.xml中修改zookeeper的属性
修改如下:
<property>
<name>hive.zookeeper.quorum</name>
<value>node1,node2,node3</value>
<description>The list of ZooKeeper servers to talk to. This is only needed for read/write locks.</description>
</property>
<property>
<name>hive.zookeeper.client.port</name>
<value>2181</value>
<description>The port of ZooKeeper servers to talk to. This is only needed for read/write locks.</description>
</property>
测试在Hive中创建表同时关联HBase
CREATE TABLE hive_hbase_emp_table(
empno int,
ename string,
job string,
mgr int,
hiredate string,
sal double,
comm double,
deptno int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,info:ename,info:job,info:mgr,info:hiredate,info:sal,info:comm,info:deptno")
TBLPROPERTIES ("hbase.table.name" = "hbase_emp_table");
提示:完成之后,可以分别进入Hive和HBase查看,都生成了对应的表
在进入hive操作hbase创建表格式:
-- 创建一张表,有两列,key int,另外一个是value string类型
CREATE TABLE hbase_table_1(key int, val string)
-- 这个代码写死
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
-- hive的表与hbase的键和值,如何映射
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
-- 在hbase中的表名
TBLPROPERTIES ("hbase.table.name" = "mydata:hbase_table_1",
"hbase.mapred.output.outputtable" = "mydata:hbase_table_1");