Flink 运维问题汇总

1.flink Slot request bulk is not fulfillable

原因 yarn资源不足了

2.flink-on-yarn模式下：Deployment took more than 60 seconds. Please check if the requested resources are

  <property>
        <name>yarn.scheduler.minimum-allocation-mb</name>
        <value>1024</value>
    </property>
    <property>
        <name>yarn.scheduler.maximum-allocation-mb</name>
        <value>10240</value>
    </property>
    <property>
        <name>yarn.nodemanager.resource.cpu-vcores</name>
        <value>32</value>
    </property>
   <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>5120</value>
    </property>

3.org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'hive' that implements 'org.apache.flink.table.factories.CatalogFactory' in the classpath

原因：flink缺少flink-connector-hive_2.12-1.14.2.jar包（需要和flink、scala版本一致）

解决办法：cp hive-exec-3.1.2.jar  /flink-1.14.2/lib/

4.java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration

cp /data/cmpt/hadoop-3.3.2/share/hadoop/common/hadoop-common-3.3.2.jar   /data/cmpt/flink-1.14.2/lib

或者

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>3.3.2</version>
</dependency>

5.java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper

cp /data/cmpt/hadoop-3.3.2/share/hadoop/common/lib/woodstox-core-5.3.0.jar /data/cmpt/flink-1.14.2/lib

6.java.lang.NoClassDefFoundError: org/codehaus/stax2/XMLInputFactory2

cp /data/cmpt/hadoop-3.3.2/share/hadoop/common/lib/stax2-api-4.2.1.jar /data/cmpt/flink-1.14.2/lib

7.NoResourceAvailableException: Could not acquire the minimum required resources.

jobmanager.memory.process.size: 6400m
taskmanager.memory.process.size: 6600m
taskmanager.numberOfTaskSlots: 50