MySQL HeatWave airportdb Quickstart 实验过程

本实验参考此文档

这个实验可以了解以下知识点:

  1. 如何从对象存储倒入数据库到MySQL
  2. 如何使用Autopilot中的Auto Parallel Load特性加载数据到HeatWave

这里提供了很多MySQL的示例数据库。我们需要的airport DB从这里下载,约640MB。

下载并解压:

$ wget https://downloads.mysql.com/docs/airport-db.zip

$ ls -l airport-db.zip
-rw-rw-r--. 1 opc opc 655695008 Sep 10 17:26 airport-db.zip

$ unzip airport-db.zip

$ ls -l airport-db
total 641612
-rwxr-xr-x. 1 opc opc     1589 Sep 10 17:25 airportdb@airline@@0.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@airline@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      627 Sep 10 17:25 airportdb@airline.json
-rwxr-xr-x. 1 opc opc     1259 Sep 10 17:25 airportdb@airline.sql
-rwxr-xr-x. 1 opc opc    24022 Sep 10 17:25 airportdb@airplane@@0.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@airplane@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      628 Sep 10 17:25 airportdb@airplane.json
-rwxr-xr-x. 1 opc opc     1105 Sep 10 17:25 airportdb@airplane.sql
-rwxr-xr-x. 1 opc opc   181412 Sep 10 17:25 airportdb@airplane_type@@0.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@airplane_type@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      605 Sep 10 17:25 airportdb@airplane_type.json
-rwxr-xr-x. 1 opc opc     1115 Sep 10 17:25 airportdb@airplane_type.sql
-rwxr-xr-x. 1 opc opc   142410 Sep 10 17:25 airportdb@airport@@0.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@airport@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc   408522 Sep 10 17:25 airportdb@airport_geo@@0.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@airport_geo@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      774 Sep 10 17:25 airportdb@airport_geo.json
-rwxr-xr-x. 1 opc opc     1366 Sep 10 17:25 airportdb@airport_geo.sql
-rwxr-xr-x. 1 opc opc      612 Sep 10 17:25 airportdb@airport.json
-rwxr-xr-x. 1 opc opc      582 Sep 10 17:25 airportdb@airport_reachable.json
-rwxr-xr-x. 1 opc opc     1010 Sep 10 17:25 airportdb@airport_reachable.sql
-rwxr-xr-x. 1 opc opc        9 Sep 10 17:25 airportdb@airport_reachable.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@airport_reachable.tsv.zst.idx
-rwxr-xr-x. 1 opc opc     1209 Sep 10 17:25 airportdb@airport.sql
-rwxr-xr-x. 1 opc opc 23990482 Sep 10 17:25 airportdb@booking@0.tsv.zst
-rwxr-xr-x. 1 opc opc      512 Sep 10 17:25 airportdb@booking@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24787816 Sep 10 17:25 airportdb@booking@10.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@10.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24605829 Sep 10 17:25 airportdb@booking@11.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@11.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24662300 Sep 10 17:25 airportdb@booking@12.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@12.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24598565 Sep 10 17:25 airportdb@booking@13.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@13.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24474339 Sep 10 17:25 airportdb@booking@14.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@14.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24659075 Sep 10 17:25 airportdb@booking@15.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@15.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24460743 Sep 10 17:25 airportdb@booking@16.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@16.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24872813 Sep 10 17:25 airportdb@booking@17.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@17.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24730510 Sep 10 17:25 airportdb@booking@18.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@18.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24664897 Sep 10 17:25 airportdb@booking@19.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@19.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 23821261 Sep 10 17:25 airportdb@booking@1.tsv.zst
-rwxr-xr-x. 1 opc opc      520 Sep 10 17:25 airportdb@booking@1.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24477378 Sep 10 17:25 airportdb@booking@20.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@20.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24603821 Sep 10 17:25 airportdb@booking@21.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@21.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24510521 Sep 10 17:25 airportdb@booking@22.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@22.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24492613 Sep 10 17:25 airportdb@booking@23.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@23.tsv.zst.idx
-rwxr-xr-x. 1 opc opc  7089904 Sep 10 17:25 airportdb@booking@@24.tsv.zst
-rwxr-xr-x. 1 opc opc      160 Sep 10 17:25 airportdb@booking@@24.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 29859055 Sep 10 17:25 airportdb@booking@2.tsv.zst
-rwxr-xr-x. 1 opc opc      648 Sep 10 17:25 airportdb@booking@2.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 22284735 Sep 10 17:25 airportdb@booking@3.tsv.zst
-rwxr-xr-x. 1 opc opc      488 Sep 10 17:25 airportdb@booking@3.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 28336025 Sep 10 17:25 airportdb@booking@4.tsv.zst
-rwxr-xr-x. 1 opc opc      616 Sep 10 17:25 airportdb@booking@4.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24829241 Sep 10 17:25 airportdb@booking@5.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@5.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24574052 Sep 10 17:25 airportdb@booking@6.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@6.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24435530 Sep 10 17:25 airportdb@booking@7.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@7.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24483962 Sep 10 17:25 airportdb@booking@8.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@8.tsv.zst.idx
-rwxr-xr-x. 1 opc opc 24687308 Sep 10 17:25 airportdb@booking@9.tsv.zst
-rwxr-xr-x. 1 opc opc      544 Sep 10 17:25 airportdb@booking@9.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      646 Sep 10 17:25 airportdb@booking.json
-rwxr-xr-x. 1 opc opc     1371 Sep 10 17:25 airportdb@booking.sql
-rwxr-xr-x. 1 opc opc    89932 Sep 10 17:25 airportdb@employee@@0.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@employee@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      883 Sep 10 17:25 airportdb@employee.json
-rwxr-xr-x. 1 opc opc     2016 Sep 10 17:25 airportdb@employee.sql
-rwxr-xr-x. 1 opc opc 10133356 Sep 10 17:25 airportdb@flight@@0.tsv.zst
-rwxr-xr-x. 1 opc opc      256 Sep 10 17:25 airportdb@flight@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      712 Sep 10 17:25 airportdb@flight.json
-rwxr-xr-x. 1 opc opc     1035 Sep 10 17:25 airportdb@flight_log.json
-rwxr-xr-x. 1 opc opc     1827 Sep 10 17:25 airportdb@flight_log.sql
-rwxr-xr-x. 1 opc opc        9 Sep 10 17:25 airportdb@flight_log.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@flight_log.tsv.zst.idx
-rwxr-xr-x. 1 opc opc   172016 Sep 10 17:25 airportdb@flightschedule@@0.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@flightschedule@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      829 Sep 10 17:25 airportdb@flightschedule.json
-rwxr-xr-x. 1 opc opc     1686 Sep 10 17:25 airportdb@flightschedule.sql
-rwxr-xr-x. 1 opc opc     1821 Sep 10 17:25 airportdb@flight.sql
-rwxr-xr-x. 1 opc opc     1199 Sep 10 17:25 airportdb.json
-rwxr-xr-x. 1 opc opc   494930 Sep 10 17:25 airportdb@passenger@@0.tsv.zst
-rwxr-xr-x. 1 opc opc        8 Sep 10 17:25 airportdb@passenger@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc  1672340 Sep 10 17:25 airportdb@passengerdetails@@0.tsv.zst
-rwxr-xr-x. 1 opc opc       32 Sep 10 17:25 airportdb@passengerdetails@@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      748 Sep 10 17:25 airportdb@passengerdetails.json
-rwxr-xr-x. 1 opc opc     1569 Sep 10 17:25 airportdb@passengerdetails.sql
-rwxr-xr-x. 1 opc opc      633 Sep 10 17:25 airportdb@passenger.json
-rwxr-xr-x. 1 opc opc     1183 Sep 10 17:25 airportdb@passenger.sql
-rwxr-xr-x. 1 opc opc      583 Sep 10 17:25 airportdb.sql
-rwxr-xr-x. 1 opc opc 35538282 Sep 10 17:25 airportdb@weatherdata@0.tsv.zst
-rwxr-xr-x. 1 opc opc     1752 Sep 10 17:25 airportdb@weatherdata@0.tsv.zst.idx
-rwxr-xr-x. 1 opc opc  4733492 Sep 10 17:25 airportdb@weatherdata@@1.tsv.zst
-rwxr-xr-x. 1 opc opc      240 Sep 10 17:25 airportdb@weatherdata@@1.tsv.zst.idx
-rwxr-xr-x. 1 opc opc      738 Sep 10 17:25 airportdb@weatherdata.json
-rwxr-xr-x. 1 opc opc     1310 Sep 10 17:25 airportdb@weatherdata.sql
-rwxr-xr-x. 1 opc opc     2586 Sep 10 17:25 @.done.json
-rwxr-xr-x. 1 opc opc      859 Sep 10 17:25 @.json
-rwxr-xr-x. 1 opc opc    46349 Sep 10 17:25 @.manifest.json
-rwxr-xr-x. 1 opc opc      240 Sep 10 17:25 @.post.sql
-rwxr-xr-x. 1 opc opc     4884 Sep 10 17:25 README.txt
-rwxr-xr-x. 1 opc opc      240 Sep 10 17:25 @.sql

安装OCI CLI,OCI Linux 7中居然已经安装好了,还是最新版本。

使用OCI CLI将目录中的数据库文件批量上传到对象存储:

$ oci os object bulk-upload -ns ocichina001 -bn airport --src-dir ~/airport-db

然后将数据库从对象存储导入,并不需要事先建库,会自动建库:

$ mysqlsh root@10.0.1.233
MySQL  10.0.1.233:33060+ ssl  JS > util.loadDump("", {threads: 16, loadIndexes: "false", osBucketName: "airport", osNamespace: "ocichina001", ignoreVersion: true})
Loading DDL and Data from OCI ObjectStorage bucket=airport, prefix='' using 16 threads.
Opening dump...
Target is MySQL 8.0.26-u1-cloud (MySQL Database Service). Dump was produced from MySQL 8.0.26-cloud
Fetching dump data from remote location...
Fetching 14 table metadata files for schema `airportdb`...
Checking for pre-existing objects...
Executing common preamble SQL
Executing DDL script for schema `airportdb`
[Worker005] Executing DDL script for `airportdb`.`flight`
[Worker003] Executing DDL script for `airportdb`.`employee`
[Worker011] Executing DDL script for `airportdb`.`weatherdata`
[Worker012] Executing DDL script for `airportdb`.`booking`
[Worker014] Executing DDL script for `airportdb`.`flight_log`
[Worker006] Executing DDL script for `airportdb`.`passengerdetails`
[Worker010] Executing DDL script for `airportdb`.`flightschedule`
[Worker002] Executing DDL script for `airportdb`.`airport_reachable`
[Worker013] Executing DDL script for `airportdb`.`airplane_type` (indexes removed for deferred creation)
[Worker009] Executing DDL script for `airportdb`.`airplane`
[Worker015] Executing DDL script for `airportdb`.`airline`
[Worker008] Executing DDL script for `airportdb`.`passenger`
[Worker007] Executing DDL script for `airportdb`.`airport`
[Worker000] Executing DDL script for `airportdb`.`airport_geo`
NOTE: [Worker007] Error processing table `airportdb`.`airport`, will retry after delay: MySQL Error 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
[Worker013] airportdb@airplane_type@@0.tsv.zst: Records: 342  Deleted: 0  Skipped: 0  Warnings: 0
[Worker008] airportdb@airline@@0.tsv.zst: Records: 113  Deleted: 0  Skipped: 0  Warnings: 0
[Worker000] airportdb@employee@@0.tsv.zst: Records: 1000  Deleted: 0  Skipped: 0  Warnings: 0
[Worker014] airportdb@airplane@@0.tsv.zst: Records: 5583  Deleted: 0  Skipped: 0  Warnings: 0
[Worker012] airportdb@airport@@0.tsv.zst: Records: 9854  Deleted: 0  Skipped: 0  Warnings: 0
[Worker002] airportdb@flightschedule@@0.tsv.zst: Records: 9881  Deleted: 0  Skipped: 0  Warnings: 0
[Worker006] airportdb@passenger@@0.tsv.zst: Records: 36095  Deleted: 0  Skipped: 0  Warnings: 0
[Worker003] airportdb@airport_geo@@0.tsv.zst: Records: 9854  Deleted: 0  Skipped: 0  Warnings: 0
[Worker005] airportdb@passengerdetails@@0.tsv.zst: Records: 36095  Deleted: 0  Skipped: 0  Warnings: 0
[Worker011] airportdb@flight@@0.tsv.zst: Records: 462553  Deleted: 0  Skipped: 0  Warnings: 0
[Worker004] airportdb@weatherdata@0.tsv.zst: Records: 4082688  Deleted: 0  Skipped: 0  Warnings: 0
[Worker004] airportdb@weatherdata@@1.tsv.zst: Records: 543744  Deleted: 0  Skipped: 0  Warnings: 0
[Worker010] airportdb@booking@3.tsv.zst: Records: 2024926  Deleted: 0  Skipped: 0  Warnings: 0
[Worker001] airportdb@booking@0.tsv.zst: Records: 2160442  Deleted: 0  Skipped: 0  Warnings: 0
[Worker009] airportdb@booking@1.tsv.zst: Records: 2159969  Deleted: 0  Skipped: 0  Warnings: 0
[Worker008] airportdb@booking@6.tsv.zst: Records: 2214973  Deleted: 0  Skipped: 0  Warnings: 0
[Worker002] airportdb@booking@10.tsv.zst: Records: 2214991  Deleted: 0  Skipped: 0  Warnings: 0
[Worker003] airportdb@booking@12.tsv.zst: Records: 2214478  Deleted: 0  Skipped: 0  Warnings: 0
[Worker013] airportdb@booking@5.tsv.zst: Records: 2215073  Deleted: 0  Skipped: 0  Warnings: 0
[Worker012] airportdb@booking@9.tsv.zst: Records: 2214959  Deleted: 0  Skipped: 0  Warnings: 0
[Worker000] airportdb@booking@7.tsv.zst: Records: 2214707  Deleted: 0  Skipped: 0  Warnings: 0
[Worker014] airportdb@booking@8.tsv.zst: Records: 2214781  Deleted: 0  Skipped: 0  Warnings: 0
[Worker005] airportdb@booking@13.tsv.zst: Records: 2215250  Deleted: 0  Skipped: 0  Warnings: 0
[Worker006] airportdb@booking@11.tsv.zst: Records: 2215122  Deleted: 0  Skipped: 0  Warnings: 0
[Worker007] airportdb@booking@4.tsv.zst: Records: 2531382  Deleted: 0  Skipped: 0  Warnings: 0
[Worker015] airportdb@booking@2.tsv.zst: Records: 2699650  Deleted: 0  Skipped: 0  Warnings: 0
[Worker011] airportdb@booking@14.tsv.zst: Records: 2215310  Deleted: 0  Skipped: 0  Warnings: 0
[Worker000] airportdb@booking@@24.tsv.zst: Records: 644485  Deleted: 0  Skipped: 0  Warnings: 0
[Worker004] airportdb@booking@15.tsv.zst: Records: 2215229  Deleted: 0  Skipped: 0  Warnings: 0
[Worker010] airportdb@booking@16.tsv.zst: Records: 2215038  Deleted: 0  Skipped: 0  Warnings: 0
[Worker001] airportdb@booking@17.tsv.zst: Records: 2214973  Deleted: 0  Skipped: 0  Warnings: 0
[Worker009] airportdb@booking@18.tsv.zst: Records: 2215177  Deleted: 0  Skipped: 0  Warnings: 0
[Worker002] airportdb@booking@20.tsv.zst: Records: 2214883  Deleted: 0  Skipped: 0  Warnings: 0
[Worker012] airportdb@booking@23.tsv.zst: Records: 2215175  Deleted: 0  Skipped: 0  Warnings: 0
[Worker008] airportdb@booking@19.tsv.zst: Records: 2214884  Deleted: 0  Skipped: 0  Warnings: 0
[Worker003] airportdb@booking@21.tsv.zst: Records: 2214070  Deleted: 0  Skipped: 0  Warnings: 0
[Worker013] airportdb@booking@22.tsv.zst: Records: 2214692  Deleted: 0  Skipped: 0  Warnings: 0
Executing common postamble SQL

37 chunks (59.50M rows, 2.03 GB) for 12 tables in 1 schemas were loaded in 2 min 6 sec (avg throughput 16.15 MB/s)
0 warnings were reported during the load.

验证数据库已创建,表已导入:

 MySQL  10.0.1.233:33060+ ssl  JS > \sql
Switching to SQL mode... Commands end with ;
 MySQL  10.0.1.233:33060+ ssl  SQL > use airportdb;
Default schema set to `airportdb`.
Fetching table and column names from `airportdb` for auto-completion... Press ^C to stop.

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > select count(*) from flight;
+----------+
| count(*) |
+----------+
|   462553 |
+----------+
1 row in set (0.0134 sec)

添加HeatWave Cluster,默认的2个节点即可。

加载数据到HeatWave,这是Autopilot的特性:

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > CALL sys.heatwave_load(JSON_ARRAY('airportdb'), NULL);
+------------------------------------------+
| INITIALIZING HEATWAVE AUTO LOAD          |
+------------------------------------------+
| Version: 1.15                            |
|                                          |
| Load Mode: normal                        |
| Load Policy: disable_unsupported_columns |
| Output Mode: normal                      |
|                                          |
+------------------------------------------+
6 rows in set (5.2867 sec)

+-----------------------------------------------------------------------------------------------+
| OFFLOAD ANALYSIS                                                                              |
+-----------------------------------------------------------------------------------------------+
| Verifying input schemas: 1                                                                    |
| User excluded items: 0                                                                        |
|                                                                                               |
| SCHEMA                       OFFLOADABLE    OFFLOADABLE     SUMMARY OF                        |
| NAME                              TABLES        COLUMNS     ISSUES                            |
| ------                       -----------    -----------     ----------                        |
| `airportdb`                           14            104     2 table(s) are partially loadable |
|                                                                                               |
| Total offloadable schemas: 1                                                                  |
|                                                                                               |
+-----------------------------------------------------------------------------------------------+
10 rows in set (5.2867 sec)

+-----------------------------------------------------------------------------------------------------------------------------+
| CAPACITY ESTIMATION                                                                                                         |
+-----------------------------------------------------------------------------------------------------------------------------+
| Default load pool for tables: TRANSACTIONAL                                                                                 |
| Default encoding for string columns: VARLEN (unless specified in the schema)                                                |
| Estimating memory footprint for 1 schema(s)                                                                                 |
|                                                                                                                             |
|                                TOTAL       ESTIMATED       ESTIMATED       TOTAL     DICTIONARY      VARLEN       ESTIMATED |
| SCHEMA                   OFFLOADABLE   HEATWAVE NODE      MYSQL NODE      STRING        ENCODED     ENCODED            LOAD |
| NAME                          TABLES       FOOTPRINT       FOOTPRINT     COLUMNS        COLUMNS     COLUMNS            TIME |
| ------                   -----------       ---------       ---------     -------     ----------     -------       --------- |
| `airportdb`                       14        4.16 GiB       56.00 MiB          35              0          35         17.00 s |
|                                                                                                                             |
| Sufficient MySQL host memory available to load all tables.                                                                  |
| Sufficient HeatWave cluster memory available to load all tables.                                                            |
|                                                                                                                             |
+-----------------------------------------------------------------------------------------------------------------------------+
13 rows in set (5.2867 sec)

+---------------------------------------------------------------------------------------------------------------------------------------+
| EXECUTING LOAD                                                                                                                        |
+---------------------------------------------------------------------------------------------------------------------------------------+
| HeatWave Load script generated                                                                                                        |
|   Retrieve load script containing 46 generated DDL command(s) using the query below:                                                  |
|   SELECT log->>"$.sql" AS "Load Script" FROM sys.heatwave_load_report WHERE type = "sql" ORDER BY id;                                 |
|                                                                                                                                       |
| Adjusting load parallelism dynamically per table                                                                                      |
| Using current parallelism of 32 thread(s) as maximum                                                                                  |
|                                                                                                                                       |
| Using SQL_MODE: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION |
|                                                                                                                                       |
| Proceeding to load 14 tables into HeatWave                                                                                            |
|                                                                                                                                       |
| Applying changes will take approximately 16.97 s                                                                                      |
|                                                                                                                                       |
+---------------------------------------------------------------------------------------------------------------------------------------+
13 rows in set (5.2867 sec)

+----------------------------------------+
| LOADING TABLE                          |
+----------------------------------------+
| TABLE (1 of 14): `airportdb`.`airline` |
| Commands executed successfully: 3 of 3 |
| Warnings encountered: 0                |
| Table loaded successfully!             |
|   Total columns loaded: 4              |
|   Table loaded using 1 thread(s)       |
|                                        |
+----------------------------------------+
7 rows in set (5.2867 sec)

+-----------------------------------------+
| LOADING TABLE                           |
+-----------------------------------------+
| TABLE (2 of 14): `airportdb`.`airplane` |
| Commands executed successfully: 3 of 3  |
| Warnings encountered: 0                 |
| Table loaded successfully!              |
|   Total columns loaded: 4               |
|   Table loaded using 1 thread(s)        |
|                                         |
+-----------------------------------------+
7 rows in set (5.2867 sec)

+----------------------------------------------+
| LOADING TABLE                                |
+----------------------------------------------+
| TABLE (3 of 14): `airportdb`.`airplane_type` |
| Commands executed successfully: 5 of 5       |
| Warnings encountered: 0                      |
| Table loaded successfully!                   |
|   Total columns loaded: 2                    |
|   Table loaded using 1 thread(s)             |
|                                              |
+----------------------------------------------+
7 rows in set (5.2867 sec)

+----------------------------------------+
| LOADING TABLE                          |
+----------------------------------------+
| TABLE (4 of 14): `airportdb`.`airport` |
| Commands executed successfully: 3 of 3 |
| Warnings encountered: 0                |
| Table loaded successfully!             |
|   Total columns loaded: 4              |
|   Table loaded using 1 thread(s)       |
|                                        |
+----------------------------------------+
7 rows in set (5.2867 sec)

+--------------------------------------------+
| LOADING TABLE                              |
+--------------------------------------------+
| TABLE (5 of 14): `airportdb`.`airport_geo` |
| Commands executed successfully: 5 of 5     |
| Warnings encountered: 0                    |
| Table loaded successfully!                 |
|   Total columns loaded: 6                  |
|   Table loaded using 1 thread(s)           |
|                                            |
+--------------------------------------------+
7 rows in set (5.2867 sec)

+--------------------------------------------------+
| LOADING TABLE                                    |
+--------------------------------------------------+
| TABLE (6 of 14): `airportdb`.`airport_reachable` |
| Commands executed successfully: 3 of 3           |
| Warnings encountered: 0                          |
| Table loaded successfully!                       |
|   Total columns loaded: 2                        |
|   Table loaded using 1 thread(s)                 |
|                                                  |
+--------------------------------------------------+
7 rows in set (5.2867 sec)

+----------------------------------------+
| LOADING TABLE                          |
+----------------------------------------+
| TABLE (7 of 14): `airportdb`.`booking` |
| Commands executed successfully: 3 of 3 |
| Warnings encountered: 0                |
| Table loaded successfully!             |
|   Total columns loaded: 5              |
|   Table loaded using 32 thread(s)      |
|                                        |
+----------------------------------------+
7 rows in set (5.2867 sec)

+-----------------------------------------+
| LOADING TABLE                           |
+-----------------------------------------+
| TABLE (8 of 14): `airportdb`.`employee` |
| Commands executed successfully: 3 of 3  |
| Warnings encountered: 0                 |
| Table loaded successfully!              |
|   Total columns loaded: 15              |
|   Table loaded using 1 thread(s)        |
|                                         |
+-----------------------------------------+
7 rows in set (5.2867 sec)

+----------------------------------------+
| LOADING TABLE                          |
+----------------------------------------+
| TABLE (9 of 14): `airportdb`.`flight`  |
| Commands executed successfully: 3 of 3 |
| Warnings encountered: 0                |
| Table loaded successfully!             |
|   Total columns loaded: 8              |
|   Table loaded using 9 thread(s)       |
|                                        |
+----------------------------------------+
7 rows in set (5.2867 sec)

+--------------------------------------------+
| LOADING TABLE                              |
+--------------------------------------------+
| TABLE (10 of 14): `airportdb`.`flight_log` |
| Commands executed successfully: 3 of 3     |
| Warnings encountered: 0                    |
| Table loaded successfully!                 |
|   Total columns loaded: 19                 |
|   Table loaded using 1 thread(s)           |
|                                            |
+--------------------------------------------+
7 rows in set (5.2867 sec)

+------------------------------------------------+
| LOADING TABLE                                  |
+------------------------------------------------+
| TABLE (11 of 14): `airportdb`.`flightschedule` |
| Commands executed successfully: 3 of 3         |
| Warnings encountered: 0                        |
| Table loaded successfully!                     |
|   Total columns loaded: 13                     |
|   Table loaded using 1 thread(s)               |
|                                                |
+------------------------------------------------+
7 rows in set (5.2867 sec)

+-------------------------------------------+
| LOADING TABLE                             |
+-------------------------------------------+
| TABLE (12 of 14): `airportdb`.`passenger` |
| Commands executed successfully: 3 of 3    |
| Warnings encountered: 0                   |
| Table loaded successfully!                |
|   Total columns loaded: 4                 |
|   Table loaded using 1 thread(s)          |
|                                           |
+-------------------------------------------+
7 rows in set (5.2867 sec)

+--------------------------------------------------+
| LOADING TABLE                                    |
+--------------------------------------------------+
| TABLE (13 of 14): `airportdb`.`passengerdetails` |
| Commands executed successfully: 3 of 3           |
| Warnings encountered: 0                          |
| Table loaded successfully!                       |
|   Total columns loaded: 9                        |
|   Table loaded using 1 thread(s)                 |
|                                                  |
+--------------------------------------------------+
7 rows in set (5.2867 sec)

+---------------------------------------------+
| LOADING TABLE                               |
+---------------------------------------------+
| TABLE (14 of 14): `airportdb`.`weatherdata` |
| Commands executed successfully: 3 of 3      |
| Warnings encountered: 0                     |
| Table loaded successfully!                  |
|   Total columns loaded: 9                   |
|   Table loaded using 18 thread(s)           |
|                                             |
+---------------------------------------------+
7 rows in set (5.2867 sec)

+-------------------------------------------------------------------------------+
| LOAD SUMMARY                                                                  |
+-------------------------------------------------------------------------------+
|                                                                               |
| SCHEMA                          TABLES       TABLES      COLUMNS         LOAD |
| NAME                            LOADED       FAILED       LOADED     DURATION |
| ------                          ------       ------      -------     -------- |
| `airportdb`                         14            0          104       5.16 s |
|                                                                               |
+-------------------------------------------------------------------------------+
6 rows in set (5.2867 sec)

Query OK, 0 rows affected (5.2867 sec)

确认以下查询可以利用HeatWave能力(engine RAPID):

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > EXPLAIN SELECT booking.price, count(*) FROM booking WHERE booking.price > 500 GROUP BY booking.price ORDER BY booking.price LIMIT 10 \G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: booking
   partitions: NULL
         type: ALL
possible_keys: NULL
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 51306483
     filtered: 33.32999801635742
        Extra: Using where; Using temporary; Using filesort; Using secondary engine RAPID
1 row in set, 1 warning (0.0054 sec)
Note (code 1003): /* select#1 */ select `airportdb`.`booking`.`price` AS `price`,count(0) AS `count(*)` from `airportdb`.`booking` where (`airportdb`.`booking`.`price` > 500.00) group by `airportdb`.`booking`.`price` order by `airportdb`.`booking`.`price` limit 10

执行查询,使用和禁用HeatWave的查询时间为0.1596和9.0713秒,性能提升为56.8倍。

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SELECT booking.price, count(*) FROM booking WHERE booking.price > 500 GROUP BY booking.price ORDER BY booking.price LIMIT 10;
+--------+----------+
| price  | count(*) |
+--------+----------+
| 500.01 |      860 |
| 500.02 |     1207 |
| 500.03 |     1135 |
| 500.04 |     1010 |
| 500.05 |     1016 |
| 500.06 |     1039 |
| 500.07 |     1002 |
| 500.08 |     1095 |
| 500.09 |     1117 |
| 500.10 |     1106 |
+--------+----------+
10 rows in set (0.1596 sec)

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SET SESSION use_secondary_engine=OFF;
Query OK, 0 rows affected (0.0004 sec)
 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SELECT booking.price, count(*) FROM booking WHERE booking.price > 500 GROUP BY booking.price ORDER BY booking.price LIMIT 10;
+--------+----------+
| price  | count(*) |
+--------+----------+
| 500.01 |      860 |
| 500.02 |     1207 |
| 500.03 |     1135 |
| 500.04 |     1010 |
| 500.05 |     1016 |
| 500.06 |     1039 |
| 500.07 |     1002 |
| 500.08 |     1095 |
| 500.09 |     1117 |
| 500.10 |     1106 |
+--------+----------+
10 rows in set (9.0713 sec)

运行以下三个可选的查询,未使用HeatWave的查询时间为(9.0642,12.7548,23.3557 ),使用HeatWave的查询时间为(0.0800,2.2155,0.2260)。性能提升分别为(113.3,5.76,103.3)倍。

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SELECT
                                             ->     booking.price,
                                             ->     count(*)
                                             -> FROM
                                             ->     booking
                                             -> WHERE
                                             ->     booking.price > 500
                                             -> GROUP BY
                                             ->     booking.price
                                             -> ORDER BY
                                             ->     booking.price
                                             -> LIMIT
                                             ->     10;
+--------+----------+
| price  | count(*) |
+--------+----------+
| 500.01 |      860 |
| 500.02 |     1207 |
| 500.03 |     1135 |
| 500.04 |     1010 |
| 500.05 |     1016 |
| 500.06 |     1039 |
| 500.07 |     1002 |
| 500.08 |     1095 |
| 500.09 |     1117 |
| 500.10 |     1106 |
+--------+----------+
10 rows in set (9.0642 sec)

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SELECT
                                             ->     airline.airlinename,
                                             ->     AVG(datediff(departure,birthdate)/365.25) as avg_age,
                                             ->     count(*) as nb_people
                                             -> FROM
                                             ->     booking, flight, airline, passengerdetails
                                             -> WHERE
                                             ->     booking.flight_id=flight.flight_id AND
                                             ->     airline.airline_id=flight.airline_id AND
                                             ->     booking.passenger_id=passengerdetails.passenger_id AND
                                             ->     country IN ("SWITZERLAND", "FRANCE", "ITALY")
                                             -> GROUP BY
                                             ->     airline.airlinename
                                             -> ORDER BY
                                             ->     airline.airlinename, avg_age
                                             -> LIMIT 10;
+----------------------+-------------+-----------+
| airlinename          | avg_age     | nb_people |
+----------------------+-------------+-----------+
| Afghanistan Airlines | 45.92612137 |     20570 |
| Albania Airlines     | 46.02485633 |     21804 |
| American Samoa Airli | 46.10098466 |     15249 |
| Angola Airlines      | 45.97214161 |     18539 |
| Argentina Airlines   | 46.03115082 |     21298 |
| Australia Airlines   | 46.07925759 |     19986 |
| Azerbaijan Airlines  | 46.01659919 |     16011 |
| Bahamas Airlines     | 46.22542566 |     22564 |
| Belarus Airlines     | 46.10818585 |     18226 |
| Bhutan Airlines      | 46.13252703 |     22873 |
+----------------------+-------------+-----------+
10 rows in set (12.7548 sec)

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SELECT
    count(*) as nb_tickets
FROM
    booking, flight, airline, airport_geo
WHERE
    booking.flight_id=flight.flight_id AND
    airline.airline_id=flight.airline_id AND
    flight.from=airport_geo.airport_id AND
    airport_geo.country = "UNITED STATES"
GROUP BY
    airline.airlinename
ORDER BY
    nb_tickets desc, airline.airlinename
LIMIT 10;                                           airline.airlinename,
                                             ->     SUM(booking.price) as price_tickets,
                                             ->     count(*) as nb_tickets
                                             -> FROM
                                             ->     booking, flight, airline, airport_geo
                                             -> WHERE
                                             ->     booking.flight_id=flight.flight_id AND
                                             ->     airline.airline_id=flight.airline_id AND
                                             ->     flight.from=airport_geo.airport_id AND
                                             ->     airport_geo.country = "UNITED STATES"
                                             -> GROUP BY
                                             ->     airline.airlinename
                                             -> ORDER BY
                                             ->     nb_tickets desc, airline.airlinename
                                             -> LIMIT 10;
+----------------------+---------------+------------+
| airlinename          | price_tickets | nb_tickets |
+----------------------+---------------+------------+
| Falkland Is Airlines |   54329614.83 |     216237 |
| Micronesia Airlines  |   53875825.72 |     214275 |
| Brazil Airlines      |   50039222.97 |     199451 |
| Cyprus Airlines      |   47466718.63 |     189493 |
| Yugoslavia Airlines  |   46905122.48 |     186865 |
| Italy Airlines       |   45731672.06 |     182152 |
| Peru Airlines        |   45742817.89 |     182111 |
| Luxembourg Airlines  |   44537553.63 |     177952 |
| Chad Airlines        |   43162776.69 |     172159 |
| Australia Airlines   |   41201251.28 |     164501 |
+----------------------+---------------+------------+
10 rows in set (23.3557 sec)

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SET SESSION use_secondary_engine=ON ;
Query OK, 0 rows affected (0.0003 sec)

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SELECT
                                             ->     booking.price,
                                             ->     count(*)
                                             -> FROM
                                             ->     booking
                                             -> WHERE
                                             ->     booking.price > 500
                                             -> GROUP BY
                                             ->     booking.price
                                             -> ORDER BY
                                             ->     booking.price
                                             -> LIMIT
                                             ->     10;
+--------+----------+
| price  | count(*) |
+--------+----------+
| 500.01 |      860 |
| 500.02 |     1207 |
| 500.03 |     1135 |
| 500.04 |     1010 |
| 500.05 |     1016 |
| 500.06 |     1039 |
| 500.07 |     1002 |
| 500.08 |     1095 |
| 500.09 |     1117 |
| 500.10 |     1106 |
+--------+----------+
10 rows in set (0.0800 sec)

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SELECT
                                             ->     airline.airlinename,
                                             ->     AVG(datediff(departure,birthdate)/365.25) as avg_age,
                                             ->     count(*) as nb_people
                                             -> FROM
                                             ->     booking, flight, airline, passengerdetails
                                             -> WHERE
                                             ->     booking.flight_id=flight.flight_id AND
                                             ->     airline.airline_id=flight.airline_id AND
                                             ->     booking.passenger_id=passengerdetails.passenger_id AND
                                             ->     country IN ("SWITZERLAND", "FRANCE", "ITALY")
                                             -> GROUP BY
                                             ->     airline.airlinename
ORDER BY
    airline.airlinename, avg_age
LIMIT 10;                                       ORDER BY
                                             ->     airline.airlinename, avg_age
                                             -> LIMIT 10;
+----------------------+-------------+-----------+
| airlinename          | avg_age     | nb_people |
+----------------------+-------------+-----------+
| Afghanistan Airlines | 45.92607116 |     20570 |
| Albania Airlines     | 46.02480595 |     21804 |
| American Samoa Airli | 46.10093492 |     15249 |
| Angola Airlines      | 45.97209191 |     18539 |
| Argentina Airlines   | 46.03110048 |     21298 |
| Australia Airlines   | 46.07920841 |     19986 |
| Azerbaijan Airlines  | 46.01654847 |     16011 |
| Bahamas Airlines     | 46.22537506 |     22564 |
| Belarus Airlines     | 46.10813567 |     18226 |
| Bhutan Airlines      | 46.13247687 |     22873 |
+----------------------+-------------+-----------+
10 rows in set (2.2155 sec)

 MySQL  10.0.1.233:33060+ ssl  airportdb  SQL > SELECT
                                             ->     airline.airlinename,
                                             ->     SUM(booking.price) as price_tickets,
                                             ->     count(*) as nb_tickets
                                             -> FROM
                                             ->     booking, flight, airline, airport_geo
                                             -> WHERE
                                             ->     booking.flight_id=flight.flight_id AND
                                             ->     airline.airline_id=flight.airline_id AND
                                             ->     flight.from=airport_geo.airport_id AND
                                             ->     airport_geo.country = "UNITED STATES"
                                             -> GROUP BY
                                             ->     airline.airlinename
                                             -> ORDER BY
                                             ->     nb_tickets desc, airline.airlinename
                                             -> LIMIT 10;
+----------------------+---------------+------------+
| airlinename          | price_tickets | nb_tickets |
+----------------------+---------------+------------+
| Falkland Is Airlines |   54329614.83 |     216237 |
| Micronesia Airlines  |   53875825.72 |     214275 |
| Brazil Airlines      |   50039222.97 |     199451 |
| Cyprus Airlines      |   47466718.63 |     189493 |
| Yugoslavia Airlines  |   46905122.48 |     186865 |
| Italy Airlines       |   45731672.06 |     182152 |
| Peru Airlines        |   45742817.89 |     182111 |
| Luxembourg Airlines  |   44537553.63 |     177952 |
| Chad Airlines        |   43162776.69 |     172159 |
| Australia Airlines   |   41201251.28 |     164501 |
+----------------------+---------------+------------+
10 rows in set (0.2260 sec)
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值