目录
2.2 分发配置yarn-site.xml到node2,node3
1 下载上传解压
https://flink.apache.org/downloads.html#flink
tar -xvf flink-1.11.0-bin-scala_2.12.tgz -C ../soft
2 配置JobManager高可用配置
2.1 修改yarn-site.xml
这需要修改hadoop 的yarn 的配置文件yarn-site.xml
<property>
<name>yarn.resourcemanager.am.max-attempts</name>
<value>4</value>
<description>
The maximum number of application master execution attempts.
</description>
</property>
应用程序主执行尝试的最大次数。这里设置为4了默认是2,主要是为了测试,其实默认就已经够了
是yarn集群上的全局配置,对运行在集群上的所有flink任务起作用;
在yarn-site.xml中配置;
yarn.application-attempts
只对当前的flink Job起作用,且不能大于yarn.resourcemanager.am.max-attempts,不然会被yarn.resourcemanager.am.max-attempts的值覆盖掉;
在flink-conf.yaml中配置(每个Job有一个 application master,每个application master 有一个 flink-conf.yaml);
2.2 分发配置yarn-site.xml到node2,node3
[liucf@node1 flink-1.11.0]$ scp /home/liucf/soft/hadoop-3.2.2/etc/hadoop/yarn-site.xml liucf@node2:/home/liucf/soft/hadoop-3.2.2/etc/hadoop/
yarn-site.xml 100% 1534 1.5MB/s 00:00
[liucf@node1 flink-1.11.0]$ scp /home/liucf/soft/hadoop-3.2.2/etc/hadoop/yarn-site.xml liucf@node3:/home/liucf/soft/hadoop-3.2.2/etc/hadoop/
yarn-site.xml
2.3 重启yarn服务
"restartyarn"){
ssh node1 "/home/liucf/soft/hadoop-3.2.2/sbin/yarn-daemon.sh stop resourcemanager"
for i in node2 node3
do
ssh $i "/home/liucf/soft/hadoop-3.2.2/sbin/yarn-daemon.sh stop nodemanager"
done
ssh node1 "/home/liucf/soft/hadoop-3.2.2/sbin/yarn-daemon.sh start resourcemanager"
for i in node2 node3
do
ssh $i "/home/liucf/soft/hadoop-3.2.2/sbin/yarn-daemon.sh start nodemanager"
done
};;
这是我自己写的启动脚本的一部分