- 前置Hadoop2.7.5集群搭建:
https://blog.csdn.net/QYHuiiQ/article/details/123055389
- 集群角色分配
Hostname | IP | Role |
hadoop01 | 192.168.126.132 | JobManager |
hadoop02 | 192.168.126.133 | TaskManager |
hadoop03 | 192.168.126.134 | TaskManager |
- 下载flink
https://archive.apache.org/dist/flink/flink-1.13.0/
将压缩包上传服务器并解压:
[root@hadoop01 wyh]# tar -zxvf flink-1.13.0-bin-scala_2.12.tgz
- 配置JobManager节点
[root@hadoop01 conf]# pwd
/usr/local/wyh/flink-1.13.0/conf
[root@hadoop01 conf]# vi flink-conf.yaml
#将下面的配置修改为JobManager的主机名
jobmanager.rpc.address: hadoop01
- 配置TaskManager节点
[root@hadoop01 conf]# pwd
/usr/local/wyh/flink-1.13.0/conf
[root@hadoop01 conf]# vi workers
[root@hadoop01 conf]# cat workers
hadoop02
hadoop03
- 将hadoop01上的flink包分发至hadoop02和haddop03机器上
[root@hadoop01 wyh]# scp -r flink-1.13.0/ hadoop02:$PWD
[root@hadoop01 wyh]# scp -r flink-1.13.0/ hadoop03:$PWD
- 启动flink集群
[root@hadoop01 bin]# pwd
/usr/local/wyh/flink-1.13.0/bin
[root@hadoop01 bin]# ./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host hadoop01.
Starting taskexecutor daemon on host hadoop02.
Starting taskexecutor daemon on host hadoop03.
- 启动后查看三台机器上的进程
访问Web:
默认情况下每个TaskManager的任务槽是1,这里我们有两个TaskManager。
停掉flinnk集群。
[root@hadoop01 bin]# pwd
/usr/local/wyh/flink-1.13.0/bin
[root@hadoop01 bin]# ./stop-cluster.sh
然后我们尝试用Yarn模式启动集群,前提是要启动Hadoop。
- 启动Hadoop集群
[root@hadoop01 hadoop-2.7.5]# pwd
/usr/local/wyh/hadoop-2.7.5
[root@hadoop01 hadoop-2.7.5]# start-all.sh
- 以Yarn模式启动Flink集群
[root@hadoop01 bin]# ./yarn-session.sh -nm test-yarn
#此处nm参数后面跟的是自定义的命名空间
启动成功:
访问UI: