当发现Broker Load导入效率不高,很可能是并发度原因,可以尝试配置如下参数:
be.conf
max_send_batch_parallelism_per_job=50
broker laod任务:
min_bytes_per_broker_scanner=218
load_parallelism=50
send_batch_parallelism=50
fe.conf 中配置
max_broker_concurrency=100
async_pending_load_task_pool_size=50
async_loading_load_task_pool_size=20
load_checker_interval_second=1
default_load_parallelism=50