Hive性能小测试

1 篇文章 0 订阅
1 篇文章 0 订阅
1. Select 6亿条数据把ID Select处理时间
hive> INSERT OVERWRITE LOCAL DIRECTORY '/home/hadoop/aaa' select id from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1

2011-07-13 19:57:35,737 Stage-1 map = 0%, reduce = 0%
2011-07-13 19:57:40,786 Stage-1 map = 10%, reduce = 0%
2011-07-13 19:57:41,801 Stage-1 map = 24%, reduce = 0%
2011-07-13 19:57:42,811 Stage-1 map = 25%, reduce = 0%

Ended Job = job_201107111406_0013
Copying data to local directory /home/hadoop/aaa
Copying data to local directory /home/hadoop/aaa
612647594 Rows loaded to /home/hadoop/aaa
OK
Time taken: 182.608 seconds


2. Hive Count时间
Hive count 过程6亿多条记录花费31秒就可以Count完

> select count(1) from a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
2011-07-13 20:01:00,272 Stage-1 map = 0%, reduce = 0%
2011-07-13 20:01:03,291 Stage-1 map = 4%, reduce = 0%
2011-07-13 20:01:04,300 Stage-1 map = 24%, reduce = 0%


Ended Job = job_201106111406_0014
OK
612647594
Time taken: 31.689 seconds
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值