Flink Sql on Hive 的生产遇到的坑以及优化（持续更新 ...）

1、HiveSource-xxxx.xxxx’s parallelism (200) is higher than the max parallelism (128). Please lower the parallelism or increase the max parallelism.

（1）报错

这是sql-cli 连接hive,查一张表报的错

[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.runtime.JobException: Vertex Source: HiveSource-xxxx.xxxx's parallelism (200) is higher than the max parallelism (128). Please lower the parallelism or increase the max parallelism.

（2）解决

只需要改动flink包下的/conf包里sql-client-defaults.yaml这个文件里的max-parallelism改为300即可

execution:
  max-parallelism: 300 

2、flink sql读取hive表时建议手动配置table.exec.hive.fallback-mapred-reader: true生效

（1）报错

不管用sql-cli，还是把sql放在代码里，执行以下sql都是下面的结果，同时报错都是报Caused by: java.lang.IllegalArgumentException。

而我用Spark Sql跑下面的Sql都是正常的。

（1）First:
SELECT vid From table_A WHERE datekey = '20210112' AND event = 'XXX' AND vid = 'aaaaaa';   (**OK**)
SELECT vid From table_A WHERE datekey = '2