ClickHouse 问题处理case2

查看ClickHouse logs目录下每天产生大量的错误日志,刚接触CH的时候,感觉日志可读性非常不友好,不像mysql那样一看就基本可以定位问题;

CH日志中包含许多无效信息;

报错日志:

2021.05.12 23:19:08.747258 [ 46720 ] {0860894a-98fe-413b-a737-bcf1c3abeedf} <Error> TCPHandler: Code: 38, e.displayText() = DB::Exception: Cannot parse date: value is too short: Cannot parse Date from String: while executing 'FUNCTION toDate(createdate : 0) -> toDate(createdate) Date : 1', Stack trace:

0. void DB::readDateTextFallback<void>(LocalDate&, DB::ReadBuffer&)::'lambda'()::operator()() const @ 0x851222e in /usr/bin/clickhouse
1. void DB::readDateTextFallback<void>(LocalDate&, DB::ReadBuffer&) @ 0x851214a in /usr/bin/clickhouse
2. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::ConvertThroughParsing<DB::DataTypeString, DB::DataTypeDate, DB::NameToDate, (DB::ConvertFromStringExceptionMode)0, (DB::ConvertFromStringParsingMode)0>::execute<void*>(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, void*) @ 0xa19ae5d in /usr/bin/clickhouse
3. bool DB::callOnIndexAndDataType<DB::DataTypeDate, DB::FunctionConvert<DB::DataTypeDate, DB::NameToDate, DB::ToDateMonotonicity>::executeInternal(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, auto const&)&, DB::ConvertDefaultBehaviorTag>(DB::TypeIndex, auto&&, DB::ToDateMonotonicity&&...) @ 0xa1902f0 in /usr/bin/clickhouse
4. DB::FunctionConvert<DB::DataTypeDate, DB::NameToDate, DB::ToDateMonotonicity>::executeInternal(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0xa18f7fc in /usr/bin/clickhouse
5. DB::FunctionConvert<DB::DataTypeDate, DB::NameToDate, DB::ToDateMonotonicity>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0xa18ee3c in /usr/bin/clickhouse
6. DB::DefaultExecutable::execute(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x9fb0c3e in /usr/bin/clickhouse
7. DB::ExecutableFunctionAdaptor::executeWithoutLowCardinalityColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0xa00dc30 in /usr/bin/clickhouse
8. DB::ExecutableFunctionAdaptor::execute(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0xa00e29d in /usr/bin/clickhouse
9. DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool) const @ 0xe9356a6 in /usr/bin/clickhouse
10. DB::ExpressionActions::execute(DB::Block&, bool) const @ 0xe936332 in /usr/bin/clickhouse
11. DB::MergeTreeDataWriter::splitBlockIntoParts(DB::Block const&, unsigned long, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf3d6cdc in /usr/bin/clickhouse
12. DB::MergeTreeBlockOutputStream::write(DB::Block const&) @ 0xf2fd518 in /usr/bin/clickhouse
13. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xeb74a8e in /usr/bin/clickhouse
14. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xeb7d10b in /usr/bin/clickhouse
15. DB::SquashingBlockOutputStream::finalize() @ 0xeb7bb6c in /usr/bin/clickhouse
16. DB::SquashingBlockOutputStream::writeSuffix() @ 0xeb7bbe9 in /usr/bin/clickhouse
17. DB::TCPHandler::processInsertQuery(DB::Settings const&) @ 0xf5c2500 in /usr/bin/clickhouse
18. DB::TCPHandler::runImpl() @ 0xf5bb129 in /usr/bin/clickhouse
19. DB::TCPHandler::run() @ 0xf5caba9 in /usr/bin/clickhouse
20. Poco::Net::TCPServerConnection::start() @ 0x11b6786f in /usr/bin/clickhouse
21. Poco::Net::TCPServerDispatcher::run() @ 0x11b69281 in /usr/bin/clickhouse
22. Poco::PooledThread::run() @ 0x11ca0329 in /usr/bin/clickhouse
23. Poco::ThreadImpl::runnableEntry(void*) @ 0x11c9c18a in /usr/bin/clickhouse
24. start_thread @ 0x7e25 in /usr/lib64/libpthread-2.17.so
25. __clone @ 0xf834d in /usr/lib64/libc-2.17.so

原因分析: 

Cannot parse date:value is too short: Cannot parse Date from String: while executing 'FUNCTION toDate(createdate : 0) -> toDate(createdate) Date : 1'

值太短,当执行toDate() 函数的时候,无法从字符串中解析到日志;

这个字段定义的是String, 里面的数据是日期时间格式; 并且表定义的时候根据该字段获取日期为分区字段;

写入的数据中,该字段可能是空值,导致获取日期函数toDate()报错,从而导致这部分数据写入失败; 

解决方法: 

方法一: 修改数据源,该值若为空,则给个当前时间;

方法二: 新增一个字段,默认取当前日期,做为PARTITION; 这样可以不需要修改数据源; 

注意: ClickHouse不支持修改PARTITION的DDL操作; 需要另外新建一个表结构,然后进行REANME, 再将老的数据导回导新表;

 

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值