hive和hbase进行关联,将hbase中的和hive中的表进行关联
1、把hive下的hive-hbase-handler-0.13.1-cdh5.3.6.jar包放到hbase的lib下,并修改hive-site.xml文件
<property>
<name>hive.zookeeper.quorum</name>
<value>linux01,linux02,linux03</value>
</property>
<property>
<name>hive.zookeeper.client.port</name>
<value>2181</value>
</property>
2、把hbsae下所有以hbase开头的包全部都放在hive的lib下
还有把hbase的 htrace-core-2.04.jar包也放到hive的lib报下
3、
在hive-site.xml文件中添加如下
注意:不能有空格!!!
<property>
<name>hive.aux.jars.path</name>
<value>file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-client-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-common-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-common-0.98.6-cdh5.3.6-tests.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-examples-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop2-compat-0.98.6-cdh5.3.6-tests.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-hadoop-compat-0.98.6-cdh5.3.6-tests.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-it-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-it-0.98.6-cdh5.3.6-tests.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-prefix-tree-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-protocol-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-server-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-server-0.98.6-cdh5.3.6-tests.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-shell-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-testing-util-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hbase-thrift-0.98.6-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hive-ant-0.13.1-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hive-beeline-0.13.1-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hive-cli-0.13.1-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hive-common-0.13.1-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/zookeeper-3.4.5-cdh5.3.6.jar,file:///home/bigdata/install/hive-0.13.1-cdh5.3.6/lib/hive-hbase-handler-0.13.1-cdh5.3.6.jar</value>
</property>
在hive中执行命令时:
hive-hbase-handler-0.13.1-cdh5.3.6.jar如果报这个jar包不存在,就把上面的这个jar包去掉再试
问题1
Caused by : java.lang.ClassNotFoundException: org.cloudera.htrace.Trace的错误解决办法
把hbase下的
问题2、
return code -101 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. org/cliffc/high_scale_lib/Counter
cp /home/bigdata/install/hbase-0.98.6-cdh5.3.6/lib/high-scale-lib-1.1.1.jar /hive的lib/
建表:
CREATE TABLE hive_hbase_callLog_table(rowkey string, dialingPhone string,
dialingName string, calledPhone string, calledName string,
callTime int,callData string)
STORED BY ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’ WITH SERDEPROPERTIES (“hbase.columns.mapping” =
“:key,info:dialing_phone,info:dialing_name,info:called_phone,info:called_name,info:call_time,
info:call_date”)
TBLPROPERTIES (“hbase.table.name” = “kafka:call_logs1”);
建外表
CREATE EXTERNAL TABLE hive_hbase_callLog_table(rowkey string, dialingPhone string,
dialingName string, calledPhone string, calledName string,
callTime int,callData string)
STORED BY ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’ WITH SERDEPROPERTIES (“hbase.columns.mapping” =
“:key,info:dialing_phone,info:dialing_name,info:called_phone,info:called_name,info:call_time,
info:call_date”)
TBLPROPERTIES (“hbase.table.name” = “kafka:call_logs1”);