hive数据通过hfile导入hbase

hive数据通过hfile导入hbase

1、原始数据表

CREATE TABLE `student`(
`name` string,
`birthday` string,
`localtion` string
)

2、创建hive表

CREATE TABLE `student_info`(
`rowkey` string,
`name` string,
`birthday` string,
`localtion` string
)
STORED AS
INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.hbase.HiveHFileOutputFormat'
TBLPROPERTIES ('hfile.family.path' = '/tmp/student/baseInfo');

3、添加jar包

• 需要将hive/lib下的所有关于hbase的jar

ll lib | grep hbase

• 在hive中

add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-client-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-common-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-common-2.0.0-alpha4-tests.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-hadoop2-compat-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-hadoop2-compat-2.0.0-alpha4-tests.jar;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-hadoop-compat-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-http-2.0.0-alpha4.jar  ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-mapreduce-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-metrics-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-metrics-api-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-prefix-tree-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-procedure-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-protocol-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-protocol-shaded-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-replication-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-server-2.0.0-alpha4.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-shaded-miscellaneous-1.0.1.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-shaded-netty-1.0.1.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hbase-shaded-protobuf-1.0.1.jar ;
add jar /home/hadoop/work/apps/hive-3.1.2/lib/hive-hbase-handler-3.1.2.jar ;

4、导入数据

insert into table student_info 
select 
concat(name, '_', birthday, '_', localtion) as rowkey,
name,
birthday,
localtion
from student
order by rowkey
;

5、查看hdfs数据

hdfs dfs -ls /tmp/student/baseInfo

6、通过bulkload导入到hbase

创建对应的hbase表，其中baseInfo为family，应和上一步对应

create 'student_info','baseInfo'

执行bulkhead，将数据导入到hbase中

bin/hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles hdfs://master:9000/tmp/student  student_info

注意hdfs目录的baseInfo为hbase的列族，bulkhead是不加

7、hbase查看数据

scan 'student_info'

注意hdfs目录的baseInfo为hbase的列族，bulkhead是不加



## 7、hbase查看数据

```shell
scan 'student_info'