Doris stream load压缩传输数据

最新推荐文章于 2024-06-11 17:40:46 发布

wangleigiser

最新推荐文章于 2024-06-11 17:40:46 发布

阅读量353

点赞数

分类专栏： Doris 文章标签：大数据 apache

本文链接：https://blog.csdn.net/wangleigiser/article/details/126743438

版权

Doris 专栏收录该内容

38 篇文章 10 订阅

订阅专栏

关于step by step 的Doris stream load参照：https://blog.csdn.net/wangleigiser/article/details/125761149?spm=1001.2014.3001.5501
CSV数据压缩

gzip -c stream_load_data.csv > stream_load_data.csv.gz

Stream Load压缩传输数据代码：

curl --location-trusted -u root:  -T stream_load_data.csv.gz -H "compress_type:GZ" -H "column_separator:," -H "label:alex-3" http://127.0.0.1:8030/api/php_doris/doris_test_sink/_stream_load

核心参数为：-H “compress_type:GZ”
执行结果：

{
    "TxnId": 9022,
    "Label": "alex-3",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 11,
    "NumberLoadedRows": 11,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 154,
    "LoadTimeMs": 27,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 1,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 7,
    "CommitAndPublishTimeMs": 18
}