1. 背景
从flink1.11开始,flink项目打包时不再强制依赖hadoop-common、hadoop-hdfs、hadoop-yarn-client等组件,虽然flink-hadoop-fs有依赖hdoop组件,如下:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<optional>true</optional>
</dependency>
但由于<optional>true</optional>
配置,导致flink-dist并不会将hadoop组件其打入其中。
另外,虽然flink-yarn中也依赖hadoop相关组件:
<dependency>
<groupId>