flink1.12部署在yarn上

1、环境介绍

CDH 6.2

flink1.12.1

2、Flink安装部署

Flink支持多种安装模式

- Local—本地单机模式，学习测试时使用

- Standalone—独立集群模式，Flink自带集群，开发测试环境使用

- StandaloneHA—独立集群高可用模式，Flink自带集群，开发测试环境使用

- On Yarn—计算资源统一由Hadoop YARN管理，生产环境使用

这里只介绍在yarn上的部署

3、flink部署到yarn上

参考：https://ci.apache.org/projects/flink/flink-docs-release-1.12/zh/deployment/resource-providers/yarn.html

Starting a Flink Session on YARN

Once you’ve made sure that the HADOOP_CLASSPATH environment variable is set, you can launch a Flink on YARN session, and submit an example job:

# we assume to be in the root directory of the unzipped Flink distribution

# (0) export HADOOP_CLASSPATH
export HADOOP_CLASSPATH=`hadoop classpath`

# (1) Start YARN Session
./bin/yarn-session.sh --detached

# (2) You can now access the Flink Web Interface through the URL printed in the last lines of the command output, or through the YARN ResourceManager web UI.

# (3) Submit example job
./bin/flink run ./examples/streaming/TopSpeedWindowing.jar

# (4) Stop YARN session (replace the application id based on the output of the yarn-session.sh command)
echo "stop" | ./bin/yarn-session.sh -id application_XXXXX_XXX

Congratulations! You have successfully run a Flink application by deploying Flink on YARN.