Hive表数据加载
五种情况
- 1、直接向分区表中插入数据
insert into table score3 partition(month =‘201807’) values (‘001’,‘002’,‘100’); - 2、通过查询插入数据
(linux ) load data local inpath ‘/export/servers/hivedatas/score.csv’ overwrite into table score partition(month=‘201806’);
(HDFS) load data inpath ‘/export/servers/hivedatas/score.csv’ overwrite into table score partition(month=‘201806’); - 3、多插入模式
from score
insert overwrite table score_first partition(month=‘201806’) select s_id,c_id
insert overwrite table score_second partition(month = ‘201806’) select c_id,s_score; - 4、查询语句中创建表并加载数据(as select)
create table score5 as select * from score; - 5、创建表时通过location指定加载数据路径
create external table score6 (s_id string,c_id string,s_score int) row format delimited fields terminated by ‘\t’ location ‘/myscore6’;
Hive数据的导出
7种方法
- 1、将查询的结果导出到本地
insert overwrite local directory ‘/export/servers/exporthive/a’ select * from score; - 2、将查询的结果格式化导出到本地
insert overwrite local directory ‘/export/servers/exporthive’ row format delimited fields terminated by ‘\t’
collection items terminated by ‘#’ select * from student; - 3、将查询的结果导出到HDFS上(没有local)
insert overwrite directory ‘/export/servers/exporthive’ row format delimited fields terminated by ‘\t’ collection items terminated by ‘#’ select * from score; - 4、Hadoop命令导出到本地
dfs -get /export/servers/exporthive/000000_0 /export/servers/exporthive/local.txt; - 5 、 hive shell 命令导出
bin/hive -e “select * from yhive.score;” > /export/servers/exporthive/score.txt - 6、export导出到HDFS上(全表导出)
export table score to ‘/export/exporthive/score’; - 7、SQOOP导出
清空hive数据表
truncate table score5;
Hive查询语句
like select * from tablename where name like ‘张%’
Rlike匹配正则
select * from tablename where name rlike ‘[]’
通配符
% 多个 100
_ 一个 1
Hive修改配置文件优先级
- 1、通过配置文件及逆行设置
- 2、hive进入shell 时,加参数 -hiveconf(命令行参数)
hive -hiveconf hive.root.logger=INFO; - 3、在Hiveshell 内进行设置(参数声明)
set mapred.reduce.tasks=100;
参数声明 > 命令行参数 > 配置文件
Hive支持的数据存储格式
- 可支持Text,
- SequenceFile,
- ParquetFile,
- ORC格式
- RCFILE等
在实际的项目开发当中,hive表的数据存储格式一般选择:orc或parquet。压缩方式一般选择snappy。