hive的测试

最新推荐文章于 2023-08-06 12:59:38 发布
最新推荐文章于 2023-08-06 12:59:38 发布
阅读量142 收藏
点赞数
文章标签： 大数据 数据库
原文链接：http://www.cnblogs.com/52hadoop/p/7690617.html
版权 
比如两个文件导入的分割符号为,
导入hive的文件
1365,fei,bai
1377,bai,360
1300,fei,360
1388,fei,360
13899,,fei
13899,fei,368
那么导入到hive里的数据
create table test3(a string,b string,c string) row format delimited fields terminated by ',' stored as textfile;
LOAD DATA LOCAL INPATH '/home/mmm/fzm/1.txt' OVERWRITE INTO TABLE test3;

hive> select  * from test3;
OK
1365    fei     bai
1377    bai     360
1300    fei     360
1388    fei     360
13899           fei
13899   fei     368
在hive 里要用=''和<>''来代替 oracle里的 is null  或者is not null  

select  * from test3 where b='';
和  select  * from test3 where b<>''

而且select  a,min(b) from test3 group by a 里取出来的是字段是空的那行，这一点和oracle不相同，oracle如果多个字段聚合，会取出不是null的那行

hive> select  * from test3;
OK
1365    fei     bai
1377    bai     360
1300    fei     360
1388    fei     360
13899           fei
13899   fei     368
Time taken: 2.157 seconds
hive> select  * from test3 where b='';
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201404230946_0174, Tracking URL = http://BJ-YZ-103R-63-33:50030/jobdetails.jsp?jobid=job_201404230946_0174
Kill Command = /apps/cloudera/parcels/CDH-4.6.0-1.cdh4.6.0.p0.26/lib/hadoop/bin/hadoop job  -kill job_201404230946_0174
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2014-05-29 10:08:21,673 Stage-1 map = 0%,  reduce = 0%
2014-05-29 10:08:27,772 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.85 sec
2014-05-29 10:08:28,812 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.85 sec
2014-05-29 10:08:29,838 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.85 sec
2014-05-29 10:08:30,862 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.85 sec
2014-05-29 10:08:31,902 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1.85 sec
MapReduce Total cumulative CPU time: 1 seconds 850 msec
Ended Job = job_201404230946_0174
MapReduce Jobs Launched: 
Job 0: Map: 1   Cumulative CPU: 1.85 sec   HDFS Read: 309 HDFS Write: 11 SUCCESS
Total MapReduce CPU Time Spent: 1 seconds 850 msec
OK
13899           fei
Time taken: 16.898 seconds
hive> select  * from test3 where b<>'';
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201404230946_0175, Tracking URL = http://BJ-YZ-103R-63-33:50030/jobdetails.jsp?jobid=job_201404230946_0175
Kill Command = /apps/cloudera/parcels/CDH-4.6.0-1.cdh4.6.0.p0.26/lib/hadoop/bin/hadoop job  -kill job_201404230946_0175
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2014-05-29 10:09:49,533 Stage-1 map = 0%,  reduce = 0%
2014-05-29 10:09:55,600 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.08 sec
2014-05-29 10:09:56,625 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.08 sec
2014-05-29 10:09:57,649 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.08 sec
2014-05-29 10:09:58,674 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2.08 sec
MapReduce Total cumulative CPU time: 2 seconds 80 msec
Ended Job = job_201404230946_0175
MapReduce Jobs Launched: 
Job 0: Map: 1   Cumulative CPU: 2.08 sec   HDFS Read: 309 HDFS Write: 66 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 80 msec
OK
1365    fei     bai
1377    bai     360
1300    fei     360
1388    fei     360
13899   fei     368
Time taken: 14.569 seconds
hive> select  a,max(b) from test3 group by a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201404230946_0176, Tracking URL = http://BJ-YZ-103R-63-33:50030/jobdetails.jsp?jobid=job_201404230946_0176
Kill Command = /apps/cloudera/parcels/CDH-4.6.0-1.cdh4.6.0.p0.26/lib/hadoop/bin/hadoop job  -kill job_201404230946_0176
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2014-05-29 10:13:15,159 Stage-1 map = 0%,  reduce = 0%
2014-05-29 10:13:21,222 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.12 sec
2014-05-29 10:13:22,245 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.12 sec
2014-05-29 10:13:23,269 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.12 sec
2014-05-29 10:13:24,293 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.12 sec
2014-05-29 10:13:25,317 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.12 sec
2014-05-29 10:13:26,341 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.12 sec
2014-05-29 10:13:27,381 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.12 sec
2014-05-29 10:13:28,404 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.8 sec
2014-05-29 10:13:29,430 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.8 sec
2014-05-29 10:13:30,455 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.8 sec
MapReduce Total cumulative CPU time: 4 seconds 800 msec
Ended Job = job_201404230946_0176
MapReduce Jobs Launched: 
Job 0: Map: 1  Reduce: 1   Cumulative CPU: 4.8 sec   HDFS Read: 309 HDFS Write: 46 SUCCESS
Total MapReduce CPU Time Spent: 4 seconds 800 msec
OK
1300    fei
1365    fei
1377    bai
1388    fei
13899   fei
Time taken: 20.759 seconds
hive> select  a,min(b) from test3 group by a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201404230946_0177, Tracking URL = http://BJ-YZ-103R-63-33:50030/jobdetails.jsp?jobid=job_201404230946_0177
Kill Command = /apps/cloudera/parcels/CDH-4.6.0-1.cdh4.6.0.p0.26/lib/hadoop/bin/hadoop job  -kill job_201404230946_0177
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2014-05-29 10:13:47,975 Stage-1 map = 0%,  reduce = 0%
2014-05-29 10:13:53,024 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.01 sec
2014-05-29 10:13:54,045 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.01 sec
2014-05-29 10:13:55,066 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.01 sec
2014-05-29 10:13:56,086 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.01 sec
2014-05-29 10:13:57,108 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.01 sec
2014-05-29 10:13:58,129 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.71 sec
2014-05-29 10:13:59,165 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.71 sec
2014-05-29 10:14:00,186 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.71 sec
2014-05-29 10:14:01,208 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.71 sec
MapReduce Total cumulative CPU time: 4 seconds 710 msec
Ended Job = job_201404230946_0177
MapReduce Jobs Launched: 
Job 0: Map: 1  Reduce: 1   Cumulative CPU: 4.71 sec   HDFS Read: 309 HDFS Write: 43 SUCCESS
Total MapReduce CPU Time Spent: 4 seconds 710 msec
OK
1300    fei
1365    fei
1377    bai
1388    fei
13899
Time taken: 18.55 seconds



hive> select  * from test3;
OK
1365    fei     bai
1377    bai     360
1300    fei     360
1388    fei     360
13899           fei
13899   fei     368

select a, count(b) from test3 group by a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201404230946_0185, Tracking URL = http://BJ-YZ-103R-63-33:50030/jobdetails.jsp?jobid=job_201404230946_0185
Kill Command = /apps/cloudera/parcels/CDH-4.6.0-1.cdh4.6.0.p0.26/lib/hadoop/bin/hadoop job  -kill job_201404230946_0185
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2014-05-29 10:39:27,598 Stage-1 map = 0%,  reduce = 0%
2014-05-29 10:39:32,650 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.91 sec
2014-05-29 10:39:33,690 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.91 sec
2014-05-29 10:39:34,713 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.91 sec
2014-05-29 10:39:35,737 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.91 sec
2014-05-29 10:39:36,760 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.91 sec
2014-05-29 10:39:37,783 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.39 sec
2014-05-29 10:39:38,823 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.39 sec
2014-05-29 10:39:39,849 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.39 sec
2014-05-29 10:39:40,874 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.39 sec
MapReduce Total cumulative CPU time: 4 seconds 390 msec
Ended Job = job_201404230946_0185
MapReduce Jobs Launched: 
Job 0: Map: 1  Reduce: 1   Cumulative CPU: 4.39 sec   HDFS Read: 309 HDFS Write: 36 SUCCESS
Total MapReduce CPU Time Spent: 4 seconds 390 msec
OK
1300    1
1365    1
1377    1
1388    1
13899   2

select a, count(distinct b) from test3 group by a;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201404230946_0186, Tracking URL = http://BJ-YZ-103R-63-33:50030/jobdetails.jsp?jobid=job_201404230946_0186
Kill Command = /apps/cloudera/parcels/CDH-4.6.0-1.cdh4.6.0.p0.26/lib/hadoop/bin/hadoop job  -kill job_201404230946_0186
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2014-05-29 10:40:14,211 Stage-1 map = 0%,  reduce = 0%
2014-05-29 10:40:21,275 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.24 sec
2014-05-29 10:40:22,297 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.24 sec
2014-05-29 10:40:23,320 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.24 sec
2014-05-29 10:40:24,340 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.24 sec
2014-05-29 10:40:25,360 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.24 sec
2014-05-29 10:40:26,382 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.24 sec
2014-05-29 10:40:27,419 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.24 sec
2014-05-29 10:40:28,440 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 5.35 sec
2014-05-29 10:40:29,463 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 5.35 sec
2014-05-29 10:40:30,485 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 5.35 sec
2014-05-29 10:40:31,506 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 5.35 sec
MapReduce Total cumulative CPU time: 5 seconds 350 msec
Ended Job = job_201404230946_0186
MapReduce Jobs Launched: 
Job 0: Map: 1  Reduce: 1   Cumulative CPU: 5.35 sec   HDFS Read: 309 HDFS Write: 36 SUCCESS
Total MapReduce CPU Time Spent: 5 seconds 350 msec
OK
1300    1
1365    1
1377    1
1388    1
13899   2

说明你不管distinct 没有  在hive里 空都算一个的 这点和oracle不一样

 

转载于:https://www.cnblogs.com/52hadoop/p/7690617.html

weixin_30575309
关注
hive测试数据
04-23
这个是hive测试数据.
大数据Hive测试数据uaction.rar
09-30
标题中的“大数据Hive测试数据uaction.rar”指的是一个用于测试Hive处理能力的数据集，这个数据集主要关注用户操作记录。Hive是Apache Hadoop生态系统中的一个组件，它提供了一个基于SQL的查询语言（HQL）来处理和...
hive学习-测试数据
weixin_34004750的博客
03-31 338
测试数据： ---------------------------------------------MySQL--------------------------------------------------- create table dept(  deptno int not null,  dname  varchar(14),  loc    varchar(13)) insert in...
hive-测试
ty-boy的博客
12-31 708
hive-测试 hive ##------命令行直接输入，如果没有这个命令，看环境变量是否添加，或者profile是否生效。 hive> show functions; OK ! != $sum0 % & * + - / < <= <=> <> = == > >= ^ abs acos add_months aes_decrypt...
Hive练习
cjwfinal的博客
09-05 325
一、创建表 1.创建students表 创建表 create table students( id bigint comment '学生id', name string comment '学生姓名', age int comment '学生年龄', gender string comment '学生性别', clazz string comment '学生班级' ) comment '学生信息表' row format delimited fields termina
11G数据的hive初测试
数据库天地
12-06 122
首先，看到标题，我想就会有人喷我，①11G对于hadoop而言不值一提②做耗时测试本没有必要，毕竟hadoop用于线下系统。 我的想法：没那么复杂，只是想看下hadoop的文件处理能力和表现出的问题，只是单纯的享受这个解决问题并且得到结果的快感。 不那么多废话啦!直接进入正题： 1、数据来源： apache的日志，接近900万条数据，大小为11个G。 2、创建表，并用正则匹配的方式导入...
hive测试题(包含万级测试数据文件)
02-19
标题 "hive测试题(包含万级测试数据文件)" 指的是一个与Hive相关的练习或测试集，其中可能包含了用于大数据分析的万级条目数据。这些数据可能被设计成模拟真实世界的业务场景，帮助学习者或者从业者检验和提升在Hive...
TPCDS测试-99条Hive基准测试流程
03-25
本篇文章将详细介绍如何进行TPCDS在Hive上的基准测试流程，以及相关的技术要点。 **一、TPCDS简介** TPCDS是一个决策支持系统的标准测试套件，包含了99个复杂的SQL查询，涵盖了多种业务场景，如数据挖掘、报表生成...
打怪升级之小白的大数据之旅--hive测试数据
06-03
"打怪升级之小白的大数据之旅--hive测试数据"这个标题，暗示了我们将通过实际操作Hive来学习和探索大数据的奥秘。Hive作为一个基于Hadoop的数据仓库工具，可以将结构化的数据文件映射为一张数据库表，并提供SQL查询...
测试hive
weixin_33728268的博客
02-15 228
2019独角兽企业重金招聘Python工程师标准>>> ...
hive 操作相关的测试数据集
11-28
hive 操作相关的测试数据集hive
Hive基础测试操作
SHZHJ2000的博客
04-10 262
一、Hive测试 　　1.查看数据库 　　show databases; 　　2.使用某个数据库，如默认数据库 　　user default; 　　3.创建表 　　create table if not exist itstar(id int,name string); 　　4.插入数据 　　insert into table itstar values(1,"wyh")...
Hive 安装与测试
Hipparchus的博客
11-30 1521
内容： 1.      安装Hive 2.      测试Hive运行情况 一、Hive 安装          首先，Hive是以来与HDFS之上的应用，所以必须先要安装Hadoop集群;这里我选择MySQL作为，所以再安装Hive前需要安装Mysql，关于MySQL和Hadoop的安装、配置这里不做详细的说明了。          这里我已经装好了Hadoop和Mysql，直
Hive 1.2.1 安装测试
易者悠也
12-23 1716
1 安装MySQL安装服务器：sudo apt-get install mysql-server(注意：期间需要设置root用户的密码哟) 安装客户端:sudo apt-get install mysql-client sudo apt-get install libmysqlclient-dev检查系统中是否有MySQL服务，出现下面的大概就是成功了sudo netstat -tap|g
Hive 的基准测试和性能测试
最新发布
互联网知识分享
08-06 593
基准测试是一种用于评估系统性能的测试方法，可以帮助我们发现系统的瓶颈和性能问题，并优化系统的配置和设计。性能测试是一种用于评估系统性能的测试方法，可以帮助我们评估系统的扩展性和可靠性，提升系统的性能和稳定性。性能测试对于评估系统的性能和稳定性非常重要，它可以帮助我们发现系统的性能瓶颈和潜在问题，并优化系统的配置和设计。基准测试对于评估系统的性能和稳定性非常重要，它可以帮助我们发现系统的瓶颈和性能问题，并优化系统的配置和设计。分析和优化：根据收集到的性能数据，分析系统的性能问题，并优化系统的配置和设计。
黑猴子的家：Hive 测试数据
黑猴子的博客
09-29 676
百度网盘分享 Hive demo 里面的测试数据 链接：https://pan.baidu.com/s/1SeEuLVJJgbuKKLVZMGcSpA 密码：opdg
Hive部署及简单测试
云来山更佳，云去山如画
05-02 4367
本篇讲述Hive的部署及简单测试
大数据组件Hive性能测试和性能优化
世界很大我很小
04-01 4298
项目介绍 这是我在华为做的最后一个项目，当时在华为海思部门。部门主要是给一些国产芯片做服务器的软件性能测试，包括大数据生态软件Hadoop、Hbase、Hive等。 测试方法 使用业界sql性能测试工具tpc-ds，选择个别SQL做性能分析优化。 TPC-DS是与真实场景非常接近的一个测试集，它包含7张事实表，17张纬度表，平均每张表含有18列。用这个数据集能够很好的模拟企业数据查询、报表生成以及数据挖掘等复杂应用。 下面是测试的数据集部分表的关系图： 测试用例举例： -- 1.符合
Hive 安装及测试一
任何技能都是从模仿开始，逐步升华。
10-16 1133
前几节演示了hadoop以及hbase的集群安装和简单实例，下面我们说一下hadoop的成员hive数据仓库的安装配置，及使用实例。hive是以及hadoop之上的，所以我们需要在hadoop的环境下，安装配置hive。
帮我创建一个hive测试表
04-27
好的，以下是创建Hive测试表的基本步骤： 1. 打开Hive终端并进入Hive交互模式。 2. 创建一个数据库： ``` CREATE DATABASE test_db; ``` 3. 使用该数据库： ``` USE test_db; ``` 4. 创建一个表： ``` CREATE...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值