【FAQ】HIVE分区表，字段类型变更后产生has an incompatible Parquet schema for column ‘table.field‘. Column type: xxxx

最新推荐文章于 2023-02-02 17:47:40 发布

loongshawn

最新推荐文章于 2023-02-02 17:47:40 发布

阅读量2.1k

点赞数

本文链接：https://blog.csdn.net/loongshawn/article/details/121946609

版权

当Hive分区表的字段类型变更后，可能会出现'has an incompatible Parquet schema for column'的错误。该异常通常由于分区文件的Parquet schema与表定义的列类型不匹配引起。解决方案包括重新写异常分区文件，如通过DROP或OVERWRITE操作。需检查并刷新受影响的历史分区数据以保持一致性。

摘要由CSDN通过智能技术生成

异常

1、常见异常

StatementCallback; uncategorized SQLException for SQL [select * from `xxx` where time= '20190404']; SQL state [HY000]; error code [0]; File 'hdfs://xxx/xxx/time=20190404/xxxxxxxxxx' has an incompatible Parquet schema for column 'xxx.xxx'. Column type: STRING, Parquet schema: optional int64 xxx [i:5 d:1 r:0] ; nested exception is java.sql.SQLException: File hdfs://xxx/xxx/time=20190404/xxxxxxxxxx has an incompatible Parquet schema for column 'xxx.xxx'. Column type: STRING, Parquet schema: optional int64 xxx [j:9 x:6 b:1]

StatementCallback; uncategorized SQLException for SQL [select * from xxx where time= ‘20190404’]; SQL state [HY000]; error code [0]; File ‘hdfs://xxx/xxx/time=20190404/xxxxxxxxxx’ has an incompatible Parquet schema for column ‘xxx.xxx’. Column type: STRING, Parquet schema: optional int64 xxx [i:5 d:1