Hive对接hbase方式之一：建立关联表

一、hive与hbase版本兼容性：
hive 1.2.x----hbase0.98.x
hive 2.x—hbase 1.x
hive 1.2.x与hbase1.x兼容要重新编译源码包hive-hbase-handler-1.2.1.jar

安装Hbase：
1 上传hbase安装到linux的路径下并解压
sudo tar -zvxf hbase-0.98.24-bin.tar.gz -C /opt/app/

2 修改权限为hadoop
sudo chown -R hadoop:hadoop hbase-0.98.24/

3 配置Hbase的环境，配置hbase/conf下的hbase-env.sh
export JAVA_HOME=/usr/java/jdk1.8.0_211-amd64
export HBASE_MANAGES_ZK=false
export HBASE_CLASSPATH=/opt/app/hbase-0.98.24-hadoop2/conf

4 配置hbase-site.xml配置文件(自定义配置文件)

5 把hdfs-site.xml 和 core-site.xml 两个配置文件复制到hbase的conf目录下
cp core-site.xml /opt/app/hbase-0.98.24/conf/
cp hdfs-site.xml /opt/app/hbase-0.98.24/conf/

6 配置regionservers, 该文件类似于 salves 文件,配置过程中存在几个regionserver 就写几个主机名称
hadoop1
hadoop2
hadoop3

7 修改 hbase/lib中关于hadoop的jar包为使用hadoop版本的jar包，（由于hbase要操作hdfs，所以hbase需要hadoop的jar包支持）。
删除原hbase /lib中的原hadoop版本jar包，从hadoop/lib 中拷贝相应hadoop jar包到hbase/lib中
cp /opt/app/hadoop-2.7.2/share/hadoop/common/lib/hadoop-annotations-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/common/lib/hadoop-auth-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/common/lib/hadoop-client-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/common/hadoop-common-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/hdfs/hadoop-hdfs-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/yarn/hadoop-yarn-client-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/
cp /opt/app/hadoop-2.7.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar /opt/app/hbase-0.98.24-hadoop2/lib/

8 分发hbase到不同的节点上
sudo scp -r hbase-0.98.24/ hadoop02:/opt/app/
sudo scp -r hbase-0.98.24/ hadoop03:/opt/app/

9 启动： hbase master服务
bin/hbase-daemon.sh start master

启动 regionserver 服务 在3个节点上启动
bin/hbase-daemon.sh start regionserver

查看WEB界面 端口60010

Hive与HBase的相互通信主要是依靠hive-hbase-handler.jar工具类 ；hive-hbase-handler.jar放在hive的lib包中而不是在hbase的lib中
添加$HIVE_HOME/conf/hive-env.sh配置属性
export HIVE_AUX_JARS_PATH=/opt/app/hbase-0.98.24-hadoop2/lib
或
修改hive的配置文件hive-site.xml，增加：

创建hive_hbase关联表：hive0.6版本以后；创建hive表的同时创建hbase表，删除 hive表的同时也会删除对应的hbase表。
create table hive_hbase_table1(ekey string, ename string, job string, sal double) partitioned by(pt string)
stored by ‘org.apache.hadoop.hive.hbase.hbasestoragehandler’
with serdeproperties (“hbase.columns.mapping” = “:key,person:name,person:job,person:sal”)
tblproperties (“hbase.table.name” = “hbase_hive_table1”,“hbase.mapred.output.outputtable” = “hbase_hive_table1”);
说明：
“:key,person:name,person:job,person:sal”----> 创建rowkey时要表示为key否则报错，有一个person列族，有name、job、sal等3列

创建hive数据表emp
create table emp1 (empkey string, ename string, job string, sal double) row format delimited fields terminated by ‘\t’;
load data local inpath ‘/data/hbasedata.txt’ overwrite into emp1 ;
insert into table hive_hbase_table1 partition(pt=‘2019-08-02’) select empkey, ename, job, sal from emp1;

创建成功！