HBase数据采集和Phoenix表映射使用索引查询的问题(解决方案)

描述:需要将数据从Oracle批量和实时采集到HBase中,并通过Phoenix映射表进行查询,支持二级索引。

二级索引类型选择使用覆盖索引。

当在Phoenix表创建二级索引之后,从Oracle采集到HBase的增量数据并没有从二级索引的条件中筛选出来,也就是Phoenix没有自动为通过HBase API进来的增量数据创建和维护索引,一般需要rebuild,方式有两种:

1. 先删除索引,再重新创建索引

2. ALTER语法

以上均从Phoenix端进行操作。

现在进行演示,数据采集工具使用DataX

创建namespace和HBase表

hbase shell:

create_namespace 'ZLXX'create 'ZLXX:WT_TRADE_REFUND', 'INFO'

 

创建schema和Phoenix表并映射到HBase表

./sqlline.py:

create schema ZLXX;create table ZLXX.WT_TRADE_REFUND (    id varchar primary key,    info.trade_id varchar,    info.amount  varchar,    info.status varchar,    info.modify_time varchar) column_encoded_bytes=0;

 

phoenix sql查询数据

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund;+-----+-----------+---------+---------+--------------+| ID  | TRADE_ID  | AMOUNT  | STATUS  | MODIFY_TIME  |+-----+-----------+---------+---------+--------------++-----+-----------+---------+---------+--------------+No rows selected (0.066 seconds)

 

初始数据采集SQL(Oracle)

select id, trade_id, amount, status, TO_CHAR(modify_time,'YYYYMMDDhh24miss') AS modify_time from zlxx.trade_refund where id in ('1215529','1215528','1205528','1195531','1195530');

 

数据采集DataX配置(hbase11xwriter

这里先使用hbase11xwriter,实际上使用的是HBase的Client API。

$ vi zlxx_test.json{  "job": {    "content": [      {        "reader": {          "name": "oraclereader",          "parameter": {            "column": [],            "connection": [              {                "fetchSize": "1024",                "jdbcUrl": [                  "$DW_ORCL_ZLXX_JDBCURL"                ],                "querySql": [                  "select id, trade_id, amount, status, TO_CHAR(modify_time,'YYYYMMDDhh24miss') AS modify_time from zlxx.trade_refund where id in ('1215529','1215528','1205528','1195531','1195530')"                ]              }            ],            "password": "$DW_ORCL_ZLXX_PASSWORD",            "username": "$DW_ORCL_ZLXX_USERNAME"          }        },        "writer": {          "name": "hbase11xwriter",          "parameter": {            "hbaseConfig": {              "hbase.zookeeper.quorum": "192.168.10.11:12181,192.168.10.12:12181,192.168.10.13:12181"            },            "table": "ZLXX:WT_TRADE_REFUND",            "mode": "normal",            "nullMode": "empty",            "rowkeyColumn": [              {                "index":0,                "type":"string"              }            ],            "column": [              {                "index":1,                "name": "INFO:TRADE_ID",                "type": "string"              },              {                "index":2,                "name": "INFO:AMOUNT",                "type": "string"              },              {                "index":3,                "name": "INFO:STATUS",                "type": "string"              },              {                "index":4,                "name": "INFO:MODIFY_TIME",                "type": "string"              }            ],            "encoding": "utf-8"          }        }      }    ],    "setting": {      "speed": {        "channel": "10"      }    }  }}

 

数据采集Shell脚本

$ vi zlxx_test.shsource ~/.bash_profilepython /zlxx/datax/bin/datax.py -j "-Xms1g -Xmx1g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/zlxx/datax/log" -p"-DDW_ORCL_ZLXX_JDBCURL=$DW_ORCL_ZLXX_JDBCURL -DDW_ORCL_ZLXX_PASSWORD=$DW_ORCL_ZLXX_PASSWORD -DDW_ORCL_ZLXX_USERNAME=$DW_ORCL_ZLXX_USERNAME" zlxx_test.json

 

执行脚本同步数据

$ sh zlxx_test.sh......任务启动时刻                    : 2020-08-10 14:29:41任务结束时刻                    : 2020-08-10 14:29:52任务总计耗时                    :                 10s任务平均流量                    :               17B/s记录写入速度                    :              0rec/s读出记录总数                    :                   5读写失败总数                    :                   0

 

 

分隔线 分隔线 分隔线 分隔线 分隔线 分隔线


  

经过以上步骤之后,查询数据:

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund limit 5;+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1195530  | Z12193932  | 5.3     | 01      | 20200721170400  || 1195531  | Z12193932  | 5       | 01      | 20200721172800  || 1205528  | Z12193989  | .03     | 03      | 20200728145130  || 1215528  | Z12194190  | .01     | 01      | 20200805181300  || 1215529  | Z12194202  | 5       | 01      | 20200807154300  |+----------+------------+---------+---------+-----------------+5 rows selected (0.111 seconds)

 

首先查看一下主键的查询执行计划和查询效果:

0: jdbc:phoenix:> explain select * from zlxx.wt_trade_refund where id='1195530';+-------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+|                                                    PLAN                                                     | EST_BYTES_READ  | EST_ROWS_READ  | EST_INFO_TS  |+-------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+| CLIENT 1-CHUNK 1 ROWS 403 BYTES PARALLEL 1-WAY ROUND ROBIN POINT LOOKUP ON 1 KEY OVER ZLXX:WT_TRADE_REFUND  | 403             | 1              | 0            |+-------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+1 row selected (0.053 seconds)

 

PARALLELx-WAY—describes how many parallel scans will be merge sorted during the operation.

ROUND ROBIN—when the query doesn’t contain ORDER BY and therefore the rows can be returned in any order, ROUND ROBIN order maximizes parallelization on the client side.

Phoenix and HBase work well when your application does point lookups and small range scans.

 

根据主键ID执行查询:

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where id='1195530';+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1195530  | Z12193932  | 5.3     | 01      | 20200721170400  |+----------+------------+---------+---------+-----------------+1 row selected (0.048 seconds)

 

 

创建索引、查看执行计划并查询数据:

没有二级索引的执行计划(FULL SCAN)

时间范围查询,FULL SCAN

0: jdbc:phoenix:> explain select * from zlxx.wt_trade_refund where modify_time>='20200801' and modify_time<'20200806';+------------------------------------------------------------------------------------------+-----------------+----------------+--------------+|                                           PLAN                                           | EST_BYTES_READ  | EST_ROWS_READ  | EST_INFO_TS  |+------------------------------------------------------------------------------------------+-----------------+----------------+--------------+| CLIENT 1-CHUNK PARALLEL 1-WAY ROUND ROBIN FULL SCAN OVER ZLXX:WT_TRADE_REFUND            | null            | null           | null         ||     SERVER FILTER BY (INFO.MODIFY_TIME >= '20200801' AND INFO.MODIFY_TIME < '20200806')  | null            | null           | null         |+------------------------------------------------------------------------------------------+-----------------+----------------+--------------+

 

带上状态,FULL SCAN

0: jdbc:phoenix:> explain select * from zlxx.wt_trade_refund where modify_time>='20200728' and modify_time<'20200806' and status='01';+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+|                                                      PLAN                                                       | EST_BYTES_READ  | EST_ROWS_READ  | EST_INFO_TS  |+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+| CLIENT 1-CHUNK PARALLEL 1-WAY ROUND ROBIN FULL SCAN OVER ZLXX:WT_TRADE_REFUND                                   | null            | null           | null         ||     SERVER FILTER BY (INFO.MODIFY_TIME >= '20200728' AND INFO.MODIFY_TIME < '20200806' AND INFO.STATUS = '01')  | null            | null           | null         |+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+2 rows selected (0.03 seconds)

 

创建覆盖索引,并满足查询全部字段数据的需求

CREATE INDEX IDX_WT_TRADE_REFUND ON ZLXX.WT_TRADE_REFUND(MODIFY_TIME,STATUS) INCLUDE(ID,TRADE_ID,AMOUNT); 0: jdbc:phoenix:> CREATE INDEX IDX_WT_TRADE_REFUND ON ZLXX.WT_TRADE_REFUND(MODIFY_TIME,STATUS) INCLUDE(ID,TRADE_ID,AMOUNT);5 rows affected (6.529 seconds)

查看执行计划,时间范围(RANGE SCAN)

0: jdbc:phoenix:> explain select * from zlxx.wt_trade_refund where modify_time>='20200801' and modify_time<'20200806';+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+|                                                      PLAN                                                       | EST_BYTES_READ  | EST_ROWS_READ  | EST_INFO_TS  |+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+| CLIENT 1-CHUNK PARALLEL 1-WAY ROUND ROBIN RANGE SCAN OVER ZLXX:IDX_WT_TRADE_REFUND ['20200801'] - ['20200806']  | null            | null           | null         |+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+1 row selected (0.035 seconds)

 

带上状态(SKIP SCAN)

Phoenix uses Skip Scan for intra-row scanning which allows for significant performance improvement over Range Scan when rows are retrieved based on a given set of keys. 0: jdbc:phoenix:> explain select * from zlxx.wt_trade_refund where modify_time>='20200728' and modify_time<'20200806' and status='01';+-------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+|                                                                PLAN                                                                 | EST_BYTES_READ  | EST_ROWS_READ  | EST_INFO_TS  |+-------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+| CLIENT 1-CHUNK PARALLEL 1-WAY ROUND ROBIN SKIP SCAN ON 1 RANGE OVER ZLXX:IDX_WT_TRADE_REFUND ['20200728','01'] - ['20200806','01']  | null            | null           | null         |+-------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+1 row selected (0.043 seconds)

 

执行查询均正常

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where modify_time>='20200801' and modify_time<'20200806';+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1215528  | Z12194190  | .01     | 01      | 20200805181300  |+----------+------------+---------+---------+-----------------+1 row selected (0.05 seconds) 0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where modify_time>='20200728' and modify_time<'20200806' and status='01';+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1215528  | Z12194190  | .01     | 01      | 20200805181300  |+----------+------------+---------+---------+-----------------+1 row selected (0.048 seconds)

 

分隔线 分隔线 分隔线 分隔线 分隔线 分隔线


 

现在使用DataX往HBase采集一条新数据和变更一条旧数据,采集SQL如下(脚本执行略)

select id, trade_id, amount, status, TO_CHAR(modify_time,'YYYYMMDDhh24miss') AS modify_time from zlxx.trade_refund where id='1195529' union all select id, trade_id, amount, '07' AS status, TO_CHAR(modify_time,'YYYYMMDDhh24miss') AS modify_time from zlxx.trade_refund where id='1215529'

 

执行同步脚本

$ sh zlxx_test.sh...任务启动时刻                    : 2020-08-10 15:21:40任务结束时刻                    : 2020-08-10 15:21:51任务总计耗时                    :                 10s任务平均流量                    :                6B/s记录写入速度                    :              0rec/s读出记录总数                    :                   2读写失败总数                    :                   0

 

不使用任何索引进行查询(查不出增量数据)

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund limit 6;+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1195530  | Z12193932  | 5.3     | 01      | 20200721170400  || 1195531  | Z12193932  | 5       | 01      | 20200721172800  || 1205528  | Z12193989  | .03     | 03      | 20200728145130  || 1215528  | Z12194190  | .01     | 01      | 20200805181300  || 1215529  | Z12194202  | 5       | 01      | 20200807154300  |+----------+------------+---------+---------+-----------------+5 rows selected (0.045 seconds)

 

Phoenix查询结果还是只有5条数据,新插入的数据是没有的,而且ID='1215529'的STATUS字段并没有变更为'07'。

 

hbase shell【新数据是有的,旧数据也是正常变更了的】:​​​​​​​

hbase(main):001:0> get 'ZLXX:WT_TRADE_REFUND','1215529'COLUMN                                          CELL                                                                                                                                        INFO:AMOUNT                                    timestamp=1597044103540, value=5                                                                                                            INFO:MODIFY_TIME                               timestamp=1597044103540, value=20200807154300                                                                                               INFO:STATUS                                    timestamp=1597044103540, value=07                                                                                                           INFO:TRADE_ID                                  timestamp=1597044103540, value=Z12194202                                                                                                   1 row(s) in 0.4170 seconds hbase(main):002:0> get 'ZLXX:WT_TRADE_REFUND','1195529'COLUMN                                          CELL                                                                                                                                        INFO:AMOUNT                                    timestamp=1597044103540, value=10                                                                                                           INFO:MODIFY_TIME                               timestamp=1597044103540, value=20200721164730                                                                                               INFO:STATUS                                    timestamp=1597044103540, value=01                                                                                                           INFO:TRADE_ID                                  timestamp=1597044103540, value=Z12193930                                                                                                   1 row(s) in 0.0210 seconds

 

使用ID查询变更的数据(正常)

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where id='1215529';+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1215529  | Z12194202  | 5       | 07      | 20200807154300  |+----------+------------+---------+---------+-----------------+1 row selected (0.07 seconds)

 

使用ID查询新增的数据(正常)

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where id='1195529';+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1195529  | Z12193930  | 10      | 01      | 20200721164730  |+----------+------------+---------+---------+-----------------+1 row selected (0.045 seconds)

 

使用时间范围查询新增的数据(查不出来)

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where modify_time>='2020072116' and modify_time<'2020072117';+-----+-----------+---------+---------+--------------+| ID  | TRADE_ID  | AMOUNT  | STATUS  | MODIFY_TIME  |+-----+-----------+---------+---------+--------------++-----+-----------+---------+---------+--------------+No rows selected (0.044 seconds)

 

这里想到了重建索引,先删除索引再创建索引,肯定是可以的(不做演示)。

下面尝试第二种方法,通过ALTER语法进行索引重建

ALTER INDEX IDX_WT_TRADE_REFUND ON ZLXX.WT_TRADE_REFUND REBUILD; 0: jdbc:phoenix:> ALTER INDEX IDX_WT_TRADE_REFUND ON ZLXX.WT_TRADE_REFUND REBUILD;6 rows affected (5.071 seconds)

 

诡异的事情发生了!!!只查出来两条数据!!!

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund limit 6;+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1195529  | Z12193930  | 10      | 01      | 20200721164730  || 1215529  | Z12194202  | 5       | 07      | 20200807154300  |+----------+------------+---------+---------+-----------------+2 rows selected (0.047 seconds)

 

时间范围查询新增数据(正常)

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where modify_time>='2020072116' and modify_time<'2020072117';+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1195529  | Z12193930  | 10      | 01      | 20200721164730  |+----------+------------+---------+---------+-----------------+1 row selected (0.037 seconds)

 

但是原来其他的数据都查不出来了!!!

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where modify_time>='20200728' and modify_time<'20200806' and status='01';+-----+-----------+---------+---------+--------------+| ID  | TRADE_ID  | AMOUNT  | STATUS  | MODIFY_TIME  |+-----+-----------+---------+---------+--------------++-----+-----------+---------+---------+--------------+No rows selected (0.037 seconds)

 

实际上数据仍然是有的

hbase(main):001:0> get 'ZLXX:WT_TRADE_REFUND','1195530'COLUMN                                          CELL                                                                                                                                        INFO:AMOUNT                                    timestamp=1597040984132, value=5.3                                                                                                          INFO:MODIFY_TIME                               timestamp=1597040984132, value=20200721170400                                                                                               INFO:STATUS                                    timestamp=1597040984132, value=01                                                                                                           INFO:TRADE_ID                                  timestamp=1597040984132, value=Z12193932                                                                                                   1 row(s) in 0.3980 seconds

 

咋整???

随着增量数据的不断增加,存量数据也会越积越多,先删除索引再创建索引显然不可行,索引rebuild也不可行。

因此,放弃HBase API和Phoenix混用吧。

 

数据入口和出口都统一从Phoenix进和出。

 

分隔线 分隔线 分隔线 分隔线 分隔线 分隔线


 

DataX是支持phoenix jdbc方式写数据的。

https://github.com/alibaba/DataX

https://github.com/alibaba/DataX/blob/master/hbase11xsqlwriter/doc/hbase11xsqlwriter.md

 

重建数据表,先不创建索引,再试试rebuild重建索引的操作。

./sqlline.py:

drop table ZLXX.WT_TRADE_REFUND;

 

hbase shell:

create 'ZLXX:WT_TRADE_REFUND', 'INFO'

 

./sqlline.py:

create table ZLXX.WT_TRADE_REFUND (    id varchar primary key,    info.trade_id varchar,    info.amount  varchar,    info.status varchar,    info.modify_time varchar) column_encoded_bytes=0;

 

数据采集DataX配置(hbase11xsqlwriter​​​​​​​

$ vi zlxx_test_phoenix.json{  "job": {    "content": [      {        "reader": {          "name": "oraclereader",          "parameter": {            "column": [],            "connection": [              {                "fetchSize": "1024",                "jdbcUrl": [                  "$DW_ORCL_ZLXX_JDBCURL"                ],                "querySql": [                  "select id, trade_id, amount, status, TO_CHAR(modify_time,'YYYYMMDDhh24miss') AS modify_time from zlxx.trade_refund where id in ('1215529','1215528','1205528','1195531','1195530')"                ]              }            ],            "password": "$DW_ORCL_ZLXX_PASSWORD",            "username": "$DW_ORCL_ZLXX_USERNAME"          }        },        "writer": {          "name": "hbase11xsqlwriter",          "parameter": {            "batchSize": "256",            "hbaseConfig": {              "hbase.zookeeper.quorum": "192.168.10.11,192.168.10.12,192.168.10.13:12181",              "zookeeper.znode.parent": "/hbase"            },            "phoenixConfig": {                "phoenix.schema.isNamespaceMappingEnabled": "true",                "phoenix.schema.mapSystemTablesToNamespace": "true"            },            "table": "ZLXX:WT_TRADE_REFUND",            "nullMode": "empty",            "column": [              "ID",              "TRADE_ID",              "AMOUNT",              "STATUS",              "MODIFY_TIME"            ],            "encoding": "utf-8"          }        }      }    ],    "setting": {      "speed": {        "channel": "10"      }    }  }}

 

shell同步脚本​​​​​​​

$ vi zlxx_test_phoenix.shsource ~/.bash_profilepython /zlxx/datax/bin/datax.py -j "-Xms1g -Xmx1g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/zlxx/datax/log" -p"-DDW_ORCL_ZLXX_JDBCURL=$DW_ORCL_ZLXX_JDBCURL -DDW_ORCL_ZLXX_PASSWORD=$DW_ORCL_ZLXX_PASSWORD -DDW_ORCL_ZLXX_USERNAME=$DW_ORCL_ZLXX_USERNAME" zlxx_test_phoenix.json

 

执行数据同步

$ sh zlxx_test_phoenix.sh...任务启动时刻                    : 2020-08-10 17:54:41任务结束时刻                    : 2020-08-10 17:55:22任务总计耗时                    :                 40s任务平均流量                    :               17B/s记录写入速度                    :              0rec/s读出记录总数                    :                   5读写失败总数                    :                   0

Phoenix jdbc写数据相对来说要慢一些?!

 

直接创建索引

0: jdbc:phoenix:> CREATE INDEX IDX_WT_TRADE_REFUND ON ZLXX.WT_TRADE_REFUND(MODIFY_TIME,STATUS) INCLUDE(ID,TRADE_ID,AMOUNT);

 

采集增量数据(SQL)

select id, trade_id, amount, status, TO_CHAR(modify_time,'YYYYMMDDhh24miss') AS modify_time from zlxx.trade_refund where id='1195529' union all select id, trade_id, amount, '07' AS status, TO_CHAR(modify_time,'YYYYMMDDhh24miss') AS modify_time from zlxx.trade_refund where id='1215529'

 

执行数据同步

$ sh zlxx_test_phoenix.sh...任务启动时刻                    : 2020-08-11 14:47:15任务结束时刻                    : 2020-08-11 14:47:31任务总计耗时                    :                 16s任务平均流量                    :                6B/s记录写入速度                    :              0rec/s读出记录总数                    :                   2读写失败总数                    :                   0

 

查询数据(正常显示全部增量变更和新增)

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund limit 6;+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1195529  | Z12193930  | 10      | 01      | 20200721164730  || 1195530  | Z12193932  | 5.3     | 01      | 20200721170400  || 1195531  | Z12193932  | 5       | 01      | 20200721172800  || 1205528  | Z12193989  | .03     | 03      | 20200728145130  || 1215528  | Z12194190  | .01     | 01      | 20200805181300  || 1215529  | Z12194202  | 5       | 07      | 20200807154300  |+----------+------------+---------+---------+-----------------+6 rows selected (0.072 seconds)

 

时间范围查询

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where modify_time>='20200721' and modify_time<'20200722';+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1195529  | Z12193930  | 10      | 01      | 20200721164730  || 1195530  | Z12193932  | 5.3     | 01      | 20200721170400  || 1195531  | Z12193932  | 5       | 01      | 20200721172800  |+----------+------------+---------+---------+-----------------+3 rows selected (0.078 seconds)

 

执行计划

时间范围RANGE SCAN

0: jdbc:phoenix:> explain select * from zlxx.wt_trade_refund where modify_time>='20200721' and modify_time<'20200722';+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+|                                                      PLAN                                                       | EST_BYTES_READ  | EST_ROWS_READ  | EST_INFO_TS  |+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+| CLIENT 1-CHUNK PARALLEL 1-WAY ROUND ROBIN RANGE SCAN OVER ZLXX:IDX_WT_TRADE_REFUND ['20200721'] - ['20200722']  | null            | null           | null         |+-----------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+1 row selected (0.079 seconds)

 

时间范围,带上状态

0: jdbc:phoenix:> select * from zlxx.wt_trade_refund where modify_time>='20200715' and modify_time<'20200809' and status in ('03', '07');+----------+------------+---------+---------+-----------------+|    ID    |  TRADE_ID  | AMOUNT  | STATUS  |   MODIFY_TIME   |+----------+------------+---------+---------+-----------------+| 1205528  | Z12193989  | .03     | 03      | 20200728145130  || 1215529  | Z12194202  | 5       | 07      | 20200807154300  |+----------+------------+---------+---------+-----------------+2 rows selected (0.052 seconds)

 

执行计划

时间范围,使用in带上状态SKIP SCAN

0: jdbc:phoenix:> explain select * from zlxx.wt_trade_refund where modify_time>='20200715' and modify_time<'20200809' and status in ('03', '07');+--------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+|                                                                 PLAN                                                                 | EST_BYTES_READ  | EST_ROWS_READ  | EST_INFO_TS  |+--------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+| CLIENT 1-CHUNK PARALLEL 1-WAY ROUND ROBIN SKIP SCAN ON 2 RANGES OVER ZLXX:IDX_WT_TRADE_REFUND ['20200715','03'] - ['20200809','07']  | null            | null           | null         |+--------------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------+--------------+1 row selected (0.043 seconds)

 

【END】

往期推荐:

 

元数据中⼼的关键⽬标和技术实现⽅案

Hive程序相关规范-有助于调优

HBase内部探险-数据模型

HBase内部探险-HBase是怎么存储数据的

HBase内部探险-一个KeyValue的历险

数据中台到底怎么建设呢?

到底什么样的企业应该建设数据中台?

数据中台到底是不是大数据的下一站?

StreamSets实时采集MySQL数据到HBase

 

 

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值