项目问题：hive中读取parquet文件的问题

问题：

hive中查询一段语句：

select
    guid,
    sessionid,
    properties,
    first_value(properties['pageid']) over(partition by guid,sessionid order by ts) as in_page,
    last_value(properties['pageid']) over(partition by guid,sessionid order by ts rows between unbounded preceding and unbounded following) as out_page,
    first_value(province) over(partition by guid,sessionid order by ts) as province,
    first_value(city) over(partition by guid,sessionid order by ts) as city,
    first_value(region) over(partition by guid,sessionid order by ts) as region,
    first_value(devicetype) over(partition by guid,sessionid order by ts) as devicetype
from dwd.event_app_detail where dt='2020-09-01' and eventid='pageView'

查询后报错

表数据：

查看表信息，只有一个字段是map类型的，其他字段都是基本类型，且存储格式发现是parquet

 show create table dwd.event_app_detail;

查看日志：

排查后发现是hive中的向量化引擎对parquent的优化导致，部分复杂数据类型不是很兼容，会有这种bug

 

解决方案：

1，把表重新导成orc格式，hive对orc格式兼容比较好

2，关闭向量化引擎优化

set hive.vectorized.execution.enabled=false;