分桶
首先要开启分桶 ,不然一顿操作猛如虎,回头一看250
set hive.enforce.bucketing=true;
创建表时 + clustered by (id) into 4 buckets 根据id分成4个桶
查询时抽样 select * from tbname tablesample (bucket 1 out of 4 on id);
create table tb_user2
(
id int,
name string,
likes array,
addrs map<string, string>
)
clustered by (id) into 4 buckets
row format
delimited
fields terminated by ‘,’
collection items terminated by ‘-’
map keys terminated by ‘:’
lines terminated by ‘\n’;
insert into table tb_user2 select id, name, likes, addrs from tb_user1;
此时hdfs 中 tb_user2 000000_0,000001_0,,000002_0,000003_0 共4个文件
问题1:分桶的作用,说的明白吗 ?
抽样查询:
select * from tb_user2 tablesample(bucket 3 out of 4 on id);
select * from tb_user2 tablesample(bucket 3 out of 8 on id);
sel