Doris Stream Load导入CSV文件

最新推荐文章于 2024-04-02 10:52:23 发布

wangleigiser

最新推荐文章于 2024-04-02 10:52:23 发布

阅读量2k

点赞数

分类专栏： Doris 大数据文章标签： mysql 数据库

本文链接：https://blog.csdn.net/wangleigiser/article/details/125761149

版权

Doris 同时被 2 个专栏收录

38 篇文章 10 订阅

订阅专栏

大数据

17 篇文章 0 订阅

订阅专栏

在doris_test数据库中创建表

CREATE TABLE `doris_test_sink` (
  `id` int NULL COMMENT "",
  `number` int NULL COMMENT "",
  `price` DECIMAL(12,2) NULL COMMENT "",
  `skuname` varchar(40) NULL COMMENT "",
  `skudesc` varchar(200) NULL COMMENT ""
) ENGINE=OLAP
DUPLICATE KEY(`id`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`id`) BUCKETS 1
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "V2"
);

构造如下CSV文件

[root@17a5da45700b /]# cat stream_load_data.csv 
10001,12,13.3, test1,test
10002,100,15.3,test2,test
10003,102,16.3,test3,test
10004,120,17.3,test4,test
10005,23,10.3, test5,test
10006,24,112.3,test6,test
10007,26,13.3, test7,test
10008,29,145.3,test8,test
10009,30,16.3, test9,test
100010,32,18.3,test10,test
100011,52,18.3,test11,test
100012,62,10.3,test12,test

执行命令导入


curl --location-trusted -u bigdata:mypss  -T stream_load_data.csv -H "column_separator:," -H "label:alex-123" http://127.0.0.1:18030/api/php_doris/doris_test_sink/_stream_load

其中：stream_load_data.csv 要导入的CSV文件，bigdata为用户名，mypss为密码，-H “column_separator:,” 表示数据以逗号分割，php_doris为数据库名称，doris_test_sink为表名称。
4. 执行结果如下

ris/doris_test_sink/_stream_load
{
    "TxnId": 2004,
    "Label": "alex-123",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 12,
    "NumberLoadedRows": 12,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 315,
    "LoadTimeMs": 41,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 2,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 3,
    "CommitAndPublishTimeMs": 33
}

数据查询和验证

mysql> 
mysql> select * from doris_test_sink;
+--------+--------+-------+---------+---------+
| id     | number | price | skuname | skudesc |
+--------+--------+-------+---------+---------+
|  10001 |     12 |  13.3 |  test1  | test    |
|  10002 |    100 |  15.3 | test2   | test    |
|  10003 |    102 |  16.3 | test3   | test    |
|  10004 |    120 |  17.3 | test4   | test    |
|  10005 |     23 |  10.3 |  test5  | test    |
|  10006 |     24 | 112.3 | test6   | test    |
|  10007 |     26 |  13.3 |  test7  | test    |
|  10008 |     29 | 145.3 | test8   | test    |
|  10009 |     30 |  16.3 |  test9  | test    |
| 100010 |     32 |  18.3 | test10  | test    |
| 100011 |     52 |  18.3 | test11  | test    |
| 100012 |     62 |  10.3 | test12  | test    |
+--------+--------+-------+---------+---------+
12 rows in set (0.02 sec)

mysql>