1、环境介绍
CDH 6.2
flink1.12.1
2、Flink安装部署
Flink支持多种安装模式
- Local—本地单机模式,学习测试时使用
- Standalone—独立集群模式,Flink自带集群,开发测试环境使用
- StandaloneHA—独立集群高可用模式,Flink自带集群,开发测试环境使用
- On Yarn—计算资源统一由Hadoop YARN管理,生产环境使用
这里只介绍在yarn上的部署
3、flink部署到yarn上
Starting a Flink Session on YARN
Once you’ve made sure that the HADOOP_CLASSPATH
environment variable is set, you can launch a Flink on YARN session, and submit an example job:
# we assume to be in the root directory of the unzipped Flink distribution
# (0) export HADOOP_CLASSPATH
export HADOOP_CLASSPATH=`hadoop classpath`
# (1) Start YARN Session
./bin/yarn-session.sh --detached
# (2) You can now access the Flink Web Interface through the URL printed in the last lines of the command output, or through the YARN ResourceManager web UI.
# (3) Submit example job
./bin/flink run ./examples/streaming/TopSpeedWindowing.jar
# (4) Stop YARN session (replace the application id based on the output of the yarn-session.sh command)
echo "stop" | ./bin/yarn-session.sh -id application_XXXXX_XXX
Congratulations! You have successfully run a Flink application by deploying Flink on YARN.