ClickHouse 问题处理case2

查看ClickHouse logs目录下每天产生大量的错误日志，刚接触CH的时候，感觉日志可读性非常不友好，不像mysql那样一看就基本可以定位问题；

CH日志中包含许多无效信息；

报错日志：

2021.05.12 23:19:08.747258 [ 46720 ] {0860894a-98fe-413b-a737-bcf1c3abeedf} <Error> TCPHandler: Code: 38, e.displayText() = DB::Exception: Cannot parse date: value is too short: Cannot parse Date from String: while executing 'FUNCTION toDate(createdate : 0) -> toDate(createdate) Date : 1', Stack trace:

0. void DB::readDateTextFallback<void>(LocalDate&, DB::ReadBuffer&)::'lambda'()::operator()() const @ 0x851222e in /usr/bin/clickhouse
1. void DB::readDateTextFallback<void>(LocalDate&, DB::ReadBuffer&) @ 0x851214a in /usr/bin/clickhouse
2. COW<DB::IColumn>::immutable_ptr<DB::IColumn> DB::ConvertThroughParsing<DB::DataTypeString, DB::DataTypeDate, DB::NameToDate, (DB::ConvertFromStringExceptionMode)0, (DB::ConvertFromStringParsingMode)0>::execute<void*>(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, void*) @ 0xa19ae5d in /usr/bin/clickhouse
3. bool DB::callOnIndexAndDataType<DB::DataTypeDate, DB::FunctionConvert<DB::DataTypeDate, DB::NameToDate, DB::ToDateMonotonicity>::executeInternal(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const::'lambda'(auto const&, auto const&)&, DB::ConvertDefaultBehaviorTag>(DB::TypeIndex, auto&&, DB::ToDateMonotonicity&&...) @ 0xa1902f0 in /usr/bin/clickhouse
4. DB::FunctionConvert<DB::DataTypeDate, DB::NameToDate, DB::ToDateMonotonicity>::executeInternal(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0xa18f7fc in /usr/bin/clickhouse
5. DB::FunctionConvert<DB::DataTypeDate, DB::NameToDate, DB::ToDateMonotonicity>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0xa18ee3c in /usr/bin/clickhouse
6. DB::DefaultExecutable::execute(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const @ 0x9fb0c3e in /usr/bin/clickhouse
7. DB::ExecutableFunctionAdaptor::executeWithoutLowCardinalityColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0xa00dc30 in /usr/bin/clickhouse
8. DB::ExecutableFunctionAdaptor::execute(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long, bool) const @ 0xa00e29d in /usr/bin/clickhouse
9. DB::ExpressionActions::execute(DB::Block&, unsigned long&, bool) const @ 0xe9356a6 in /usr/bin/clickhouse
10. DB::ExpressionActions::execute(DB::Block&, bool) const @ 0xe936332 in /usr/bin/clickhouse
11. DB::MergeTreeDataWriter::splitBlockIntoParts(DB::Block const&, unsigned long, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf3d6cdc in /usr/bin/clickhouse
12. DB::MergeTreeBlockOutputStream::write(DB::Block const&) @ 0xf2fd518 in /usr/bin/clickhouse
13. DB::PushingToViewsBlockOutputStream::write(DB::Block const&) @ 0xeb74a8e in /usr/bin/clickhouse
14. DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0xeb7d10b in /usr/bin/clickhouse
15. DB::SquashingBlockOutputStream::finalize() @ 0xeb7bb6c in /usr/bin/clickhouse
16. DB::SquashingBlockOutputStream::writeSuffix() @ 0xeb7bbe9 in /usr/bin/clickhouse
17. DB::TCPHandler::processInsertQuery(DB::Settings const&) @ 0xf5c2500 in /usr/bin/clickhouse
18. DB::TCPHandler::runImpl() @ 0xf5bb129 in /usr/bin/clickhouse
19. DB::TCPHandler::run() @ 0xf5caba9 in /usr/bin/clickhouse
20. Poco::Net::TCPServerConnection::start() @ 0x11b6786f in /usr/bin/clickhouse
21. Poco::Net::TCPServerDispatcher::run() @ 0x11b69281 in /usr/bin/clickhouse
22. Poco::PooledThread::run() @ 0x11ca0329 in /usr/bin/clickhouse
23. Poco::ThreadImpl::runnableEntry(void*) @ 0x11c9c18a in /usr/bin/clickhouse
24. start_thread @ 0x7e25 in /usr/lib64/libpthread-2.17.so
25. __clone @ 0xf834d in /usr/lib64/libc-2.17.so

原因分析： 

Cannot parse date:value is too short: Cannot parse Date from String: while executing 'FUNCTION toDate(createdate : 0) -> toDate(createdate) Date : 1'

值太短，当执行toDate() 函数的时候，无法从字符串中解析到日志；

这个字段定义的是String， 里面的数据是日期时间格式； 并且表定义的时候根据该字段获取日期为分区字段；

写入的数据中，该字段可能是空值，导致获取日期函数toDate()报错，从而导致这部分数据写入失败； 

解决方法： 

方法一： 修改数据源，该值若为空，则给个当前时间；

方法二： 新增一个字段，默认取当前日期，做为PARTITION； 这样可以不需要修改数据源； 

注意： ClickHouse不支持修改PARTITION的DDL操作； 需要另外新建一个表结构，然后进行REANME， 再将老的数据导回导新表；