impala和mysql对比_hive和impala查询数据对比

最新推荐文章于 2023-06-29 04:21:35 发布

勤劳课代表

最新推荐文章于 2023-06-29 04:21:35 发布

阅读量1.2k

点赞数

文章标签： impala和mysql对比

本文链接：https://blog.csdn.net/weixin_33747775/article/details/113976760

版权

首先impala查询数据，更像rdbms一样(mysql)。

--1.impala连接

[root@MASTER01 ~]# impala-shell -islave03;

Starting Impala Shell without Kerberos authentication

Connected to slave03:21000

Server version: impalad version 2.0.0-cdh5 RELEASE (build ecf30af0b4d6e56ea80297df2189367ada6b7da7)

Welcome to the Impala shell. Press TAB twice to see a list of available commands.

(Shell build version: Impala Shell v2.0.0-cdh5 (ecf30af) built on Sat Oct 11 13:56:06 PDT 2014)

[slave03:21000] > use tmp;

Query: use tmp

[slave03:21000] > select count(*) from view_0809_02;

Query: select count(*) from view_0809_02

+----------+

| count(*) |

+----------+

| 312923 |

+----------+

Fetched 1 row(s) in 0.52s

[slave03:21000] >

[slave03:21000] > show databases;

Query: show databases

+------------------+

| name |

+------------------+

| _impala_builtins |

| analyse |

| default |

| preparation |

| result |

| tmp |

| trans |

| unicomidmp |

| unicomidmptext |

+------------------+

Fetched 9 row(s) in 0.04s

[slave03:21000] >

--2.hive查询

把你的查询写入到文件中，这里以test为例

[root@MASTER01 ~]# pwd

/root

[root@MASTER01 ~]# more test

select * from tmp.view_0809_02 limit 10;

[root@MASTER01 ~]#

方式一

[root@MASTER01 ~]# hive

2016-07-10 23:13:23,618 WARN [main] conf.HiveConf (HiveConf.java:initialize(1488)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/jars/hive-common-0.13.1-cdh5.2.0.jar!/hive-log4j.properties

hive> source /root/test;

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

SLF4J: Defaulting to no-operation (NOP) logger implementation

SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

127127010.0235272.00.00.01995.71217179076620.54606802586760131NULL

1271270410597.17747451898315851.00.00.02214.8119594275830.5935170862044721甯..

127127019000.21532129144515180.00.00.01635.47218477398880.53046095126247791甯..

127127019127.89991113976612088.00.00.01315.33129526608580.40749257449458661甯..

127127097248.96671687159712211.00.00.01917.68722227062810.477744807121661731甯..

127127017173.39387186049913058.00.00.01618.20317881411140.49416409200137321甯..

127127026611.04109938432414520.00.00.02039.50768260972340.51968944533818281甯..

127127017141.8042285512545984.00.00.01666.41422109443970.465080583269378341甯..

127127017299.3800899960093047.00.00.01652.15920627595760.50773088173840361甯..

127127036979.5982123612275697.00.00.01696.50900443295150.46532999164578111甯..

Time taken: 1.225 seconds, Fetched: 10 row(s)

hive>

方式二

[root@MASTER01 ~]# hive -f test

2016-07-10 23:14:20,138 WARN [main] conf.HiveConf (HiveConf.java:initialize(1488)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/jars/hive-common-0.13.1-cdh5.2.0.jar!/hive-log4j.properties

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

SLF4J: Defaulting to no-operation (NOP) logger implementation

SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

127127010.0235272.00.00.01995.71217179076620.54606802586760131NULL

1271270410597.17747451898315851.00.00.02214.8119594275830.5935170862044721市区

127127019000.21532129144515180.00.00.01635.47218477398880.53046095126247791市区

127127019127.89991113976612088.00.00.01315.33129526608580.40749257449458661市区

127127097248.96671687159712211.00.00.01917.68722227062810.477744807121661731市区

127127017173.39387186049913058.00.00.01618.20317881411140.49416409200137321市区

127127026611.04109938432414520.00.00.02039.50768260972340.51968944533818281市区

127127017141.8042285512545984.00.00.01666.41422109443970.465080583269378341市区

127127017299.3800899960093047.00.00.01652.15920627595760.50773088173840361市区

127127036979.5982123612275697.00.00.01696.50900443295150.46532999164578111市区

Time taken: 1.423 seconds, Fetched: 10 row(s)

Jul 10, 2016 11:14:23 PM WARNING: parquet.hadoop.ParquetRecordReader: Can not initialize counter due to context is not a instance of TaskInputOutputContext, but is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl

Jul 10, 2016 11:14:23 PM INFO: parquet.hadoop.InternalParquetRecordReader: RecordReader initialized will read a total of 23383 records.

Jul 10, 2016 11:14:23 PM INFO: parquet.hadoop.InternalParquetRecordReader: at row 0. reading next block

Jul 10, 2016 11:14:23 PM INFO: parquet.hadoop.InternalParquetRecordReader: block read in memory in 29 ms. row count = 23383

[root@MASTER01 ~]#

方式三，直接用sql

[root@MASTER01 ~]# hive -e 'select * from tmp.view_0809_02 limit 10'

2016-07-10 23:31:35,744 WARN [main] conf.HiveConf (HiveConf.java:initialize(1488)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.