本文部分引用其它博客,部分原创,如有不妥之处,欢迎随时留言沟通交流,谢谢~
1、Impala语法
增删改查
2、Impala常用函数
2.1 、字符串函数
参考链接: https://blog.csdn.net/qq_24699959/article/details/79863664
2.2 、 数值函数
参考链接: https://blog.csdn.net/qq_24699959/article/details/80090050
2.3 、 其它函数
参考链接: https://blog.csdn.net/qq_24699959/article/details/80162946
3、Impala+Kudu
3.1、Kudu分区调优
4、Imapla+HDFS
4.1、外表读HDFS文件
HDFS创建分区结构
[root@master01 ~]# hdfs dfs -ls /user/hive/warehouse/ods_logs.db/profile_dim_info
Found 5 items
drwxr-xr-x - root hive 0 2018-07-10 10:59 /user/hive/warehouse/ods_logs.db/profile_dim_info/dim_type=ip_guisd
drwxrwxrwx - impala hive 0 2018-07-03 20:32 /user/hive/warehouse/ods_logs.db/profile_dim_info/dim_type=phone_guisd
drwxrwxrwx - impala hive 0 2018-07-05 15:13 /user/hive/warehouse/ods_logs.db/profile_dim_info/dim_type=privi_type
drwxr-xr-x - root hive 0 2018-07-03 20:35 /user/hive/warehouse/ods_logs.db/profile_dim_info/dim_type=source_name
drwxrwxrwx - impala hive 0 2018-07-05 15:12 /user/hive/warehouse/ods_logs.db/profile_dim_info/dim_type=subject_type
Impala创建external表
[master01:25003] > show create table profile_dim_info;
Query: show create table profile_dim_info
+-----------------------------------------------------------------------------------+
| result |
+-----------------------------------------------------------------------------------+
| CREATE EXTERNAL TABLE ods_logs.profile_dim_info ( |
| feat1 STRING COMMENT '1', |
| feat2 STRING COMMENT '1', |
| feat3 STRING COMMENT '1', |
| feat4 STRING COMMENT '1', |
| feat5 STRING COMMENT '1', |
| feat6 STRING COMMENT '1', |
| feat7 STRING COMMENT '1', |
| feat8 STRING COMMENT '1', |
| feat9 STRING COMMENT '1', |
| feat10 STRING COMMENT '1' |
| ) |
| PARTITIONED BY ( |
| dim_type STRING |
| ) |
| ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' |
| WITH SERDEPROPERTIES ('serialization.format'='|', 'field.delim'='|') |
| STORED AS TEXTFILE |
| LOCATION 'hdfs://bigdata01:8020/user/hive/warehouse/ods_logs.db/profile_dim_info' |
| TBLPROPERTIES ('STATS_GENERATED_VIA_STATS_TASK'='true', 'numRows'='616432') |
+-----------------------------------------------------------------------------------+
Fetched 1 row(s) in 0.02s
Impala中refresh表
[master01:25003] > refresh profile_dim_info;
[master01:25003] > show partitions profile_dim_info;
+--------------+--------+--------+---------+--------------+-------------------+--------+
| dim_type | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format |
+--------------+--------+--------+---------+--------------+-------------------+--------+
| ip_guisd | 255758 | 1 | 13.14MB | NOT CACHED | NOT CACHED | TEXT |
| phone_guisd | 360563 | 1 | 14.52MB | NOT CACHED | NOT CACHED | TEXT |
| privi_type | 24 | 1 | 796B | NOT CACHED | NOT CACHED | TEXT |
| source_name | 63 | 1 | 1.21KB | NOT CACHED | NOT CACHED | TEXT |
| subject_type | 24 | 1 | 645B | NOT CACHED | NOT CACHED | TEXT |
| Total | 616432 | 5 | 27.66MB | 0B | | |
+--------------+--------+--------+---------+--------------+-------------------+--------+
Fetched 6 row(s) in 0.36s