pyflink执行任务问题总结
- 问题:
-
- 一、Timeout expired while fetching topic metadata
- 二、Deployment took more than 60 seconds
- 三、SyntaxError: invalid syntax
- 四、Failed to deserialize JobGraph
- 五、The cluster does not have the requested resources for the JobManager available!
- 六、Permission denied: user=root, access=WRITE, inode="/user":hadoop:supergroup:drwxr-xr-x
- 七、Timeout expired while fetching topic metadata
问题:
一、Timeout expired while fetching topic metadata
yarn模式下报错:
Caused by: org.apache.flink.kafka.shaded.org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
原因:内存不足
默认不设置的情况下最大为8G,最小为1G,
修改hadoop/etc/hadoop/yarn-site.xml中的value值
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>1024</value>
<description>一个container可申请的最小内存</description>
</property>