【ClickHouse系列】如何解决 Too Many Parts 问题

最新推荐文章于 2025-03-06 20:15:18 发布

一只努力的微服务

最新推荐文章于 2025-03-06 20:15:18 发布

阅读量3.1k

点赞数

分类专栏： ClickHouse 文章标签： clickhouse c++ java

本文链接：https://blog.csdn.net/weixin_39992480/article/details/129470168

版权

ClickHouse 专栏收录该内容

45 篇文章

订阅专栏

文章讲述了在MergeTree表引擎中，当INSERT操作导致partition中的active或inactiveparts超过特定阈值时，系统会抛出异常或延迟INSERT。参数如`max_parts_in_total`,`parts_to_throw_insert`,`part_to_delay_insert`等控制这些行为。当INSERT被延迟时，其延迟时间由一个公式计算得出。解决方案包括调整设置、改变插入频率和批量大小，以及启用WAL模式。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

几个相关参数

max_parts_in_total

如果一个table的所有partition中的active的part总数超过该值，会在INSERT时报Too many parts异常

默认值：100000

parts_to_throw_insert

如果单个partition中active的part总数超过该值，会在INSERT时报Too many parts异常

默认值：300

part_to_delay_insert

如果单个partition中active的part总数超过该值，则INSERT人为地减慢

默认值：150

inactive_parts_to_throw_insert

如果单个partition中inactive的part总数超过该值，会在INSERT时报Too many inactive parts 异常

默认值：0（无限制）

inactive_parts_to_delay_insert

如果表中单个partition中的inactive的part总数达到该值，则`INSERT会被减慢

默认值：0（无限制）

max_delay_to_insert

以秒为单位的值，用于计算INSERT延迟

默认值：1

计算公式如下：

max_k = parts_to_throw_insert - parts_to_delay_insert
k = 1 + parts_count_in_partition - parts_to_delay_insert
delay_milliseconds = pow(max_delay_to_insert * 1000, k / max_k)

例如，如果一个partition有 299 个active的part，parts_to_throw_insert = 300，parts_to_delay_insert = 150，max_delay_to_insert = 1，INSERT会延迟pow( 1 * 1000, (1 + 299 - 150) / (300 - 150) ) = 1000毫秒。

相关代码

在数据写入前会调用writePrefix()检查是否要抛Too many parts异常

void MergeTreeBlockOutputStream::writePrefix()
{
    /// Only check "too many parts" before write,
    /// because interrupting long-running INSERT query in the middle is not convenient for users.
    storage.delayInsertOrThrowIfNeeded();
}

MergeTree系列的实现是如下方法

void MergeTreeData::delayInsertOrThrowIfNeeded(Poco::Event * until) const
{
    const auto settings = getSettings();
    const size_t parts_count_in_total = getPartsCount();
    
    /// max_parts_in_total默认100000
    if (parts_count_in_total >= settings->max_parts_in_total)
    {
        ProfileEvents::increment(ProfileEvents::RejectedInserts);
        throw Exception("Too many parts (" + toString(parts_count_in_total) + ") in all partitions in total. This indicates wrong choice of partition key. The threshold can be modified with 'max_parts_in_total' setting in <merge_tree> element in config.xml or with per-table setting.", ErrorCodes::TOO_MANY_PARTS);
    }

    size_t parts_count_in_partition = getMaxPartsCountForPartition();
    ssize_t k_inactive = -1;
    if (settings->inactive_parts_to_throw_insert > 0 || settings->inactive_parts_to_delay_insert > 0)
    {
        size_t inactive_parts_count_in_partition = getMaxInactivePartsCountForPartition();
        if (inactive_parts_count_in_partition >= settings->inactive_parts_to_throw_insert)
        {
            ProfileEvents::increment(ProfileEvents::RejectedInserts);
            throw Exception(
                ErrorCodes::TOO_MANY_PARTS,
                "Too many inactive parts ({}). Parts cleaning are processing significantly slower than inserts",
                inactive_parts_count_in_partition);
        }
        k_inactive = ssize_t(inactive_parts_count_in_partition) - ssize_t(settings->inactive_parts_to_delay_insert);
    }

    /// parts_to_throw_insert默认300，单partition中的part超过该值会报错
    if (parts_count_in_partition >= settings->parts_to_throw_insert)
    {
        ProfileEvents::increment(ProfileEvents::RejectedInserts);
        throw Exception(
            ErrorCodes::TOO_MANY_PARTS,
            "Too many parts ({}). Parts cleaning are processing significantly slower than inserts",
            parts_count_in_partition);
    }

    if (k_inactive < 0 && parts_count_in_partition < settings->parts_to_delay_insert)
        return;

    /// 根据active的和inactive的part来计算要将下一个insert延迟多久
    const ssize_t k_active = ssize_t(parts_count_in_partition) - ssize_t(settings->parts_to_delay_insert);
    size_t max_k;
    size_t k;
    if (k_active > k_inactive)
    {
        max_k = settings->parts_to_throw_insert - settings->parts_to_delay_insert;
        k = k_active + 1;
    }
    else
    {
        max_k = settings->inactive_parts_to_throw_insert - settings->inactive_parts_to_delay_insert;
        k = k_inactive + 1;
    }
    const double delay_milliseconds = ::pow(settings->max_delay_to_insert * 1000, static_cast<double>(k) / max_k);

    ProfileEvents::increment(ProfileEvents::DelayedInserts);
    ProfileEvents::increment(ProfileEvents::DelayedInsertsMilliseconds, delay_milliseconds);

    CurrentMetrics::Increment metric_increment(CurrentMetrics::DelayedInserts);

    LOG_INFO(log, "Delaying inserting block by {} ms. because there are {} parts", delay_milliseconds, parts_count_in_partition);

    /// 通过sleep来减缓insert速度
    if (until)
        until->tryWait(delay_milliseconds);
    else
        std::this_thread::sleep_for(std::chrono::milliseconds(static_cast<size_t>(delay_milliseconds)));
}

解决方法

增加parts_to_throw_insert，有时候业务高峰时会将单partition中的part撑到几百上千后就不会再增长了，所以可以通过调大parts_to_throw_insert来解决；
业务方减小频次、增大每批数据总数，即大批量小频率写数据；
业务方insert时尽量根据partition排序写入，避免每个insert都包含很多parttition，若partition过多，会生成大于等于分区数都part，虽然数据量增大了，频率低了，但是也不能对too many part有太多缓解。
开启wal模式，减少小part的生成