flink下载镜像地址:https://dlcdn.apache.org/flink/flink-1.12.5/
官方下载地址:https://flink.apache.org/zh/downloads.html
flink官网地址
英文:https://flink.apache.org/
中文 :https://flink.apache.org/zh/
flink教程:https://nightlies.apache.org/flink/flink-docs-release-1.14/zh//docs/try-flink/local_installation/
flink中文社区:https://flink-learning.org.cn/
环境:java +flink-1.12.5-bin-scala_2.12.tgz+zookeeper3.6.2+ssh免密+关闭防火墙
3台服务器:
192.168.189.9 master StandaloneSessionClusterEntrypoint taskManagerRunner
192.168.189.10 slave1 taskManagerRunner
192.168.189.11.slave2 taskManagerRunner
安装方式:
1.上传文件flink-1.12.5-bin-scala_2.12.tgz至soft文件夹下(默认挂载最大盘)并解压
tar -zxvf flink-1.12.5-bin-scala_2.12.tgz
删除文件包
rm -rf flink-1.12.5-bin-scala_2.12.tgz
重命名
mv flink-1.12.5 flink
解压状态如下:
2.配置环境变量(可省略,在对应安装目录执行命令)
echo "#flink" >> /etc/profile
echo "FLINK_HOME=/soft/flink" >> /etc/profile
echo "PATH=$FLINK_HOME/bin:$PATH" >> /etc/profile
3.修改配置文件,三台机器,登录master修改配置文件进入目录:cd /soft/flink/conf
#leader主机配置
vi masters
master:8081
#从机配置
vi workers
master
slave1
slave2
#修改配置文件flink-conf.yaml
vi flink-conf.yaml
jobmanager.rpc.address: master
jobmanager.memory.process.size: 1024m
taskmanager.memory.process.size: 1024m
#webuiIP端口
rest.port: 8081
#修改zoo.cfg
dataDir=/soft/apache-zookeeper-3.6.2/zkdata
server.1=master:2888:3888
server.2=slave1:2888:3888
server.3=slave2:2888:3888
4.分发配置(xsync) 脚本如下
#!/bin/bash
#1 获取输入参数个数,如果没有参数,直接退出
pcount=$#
if((pcount==0)); then
echo no args;
exit;
fi
#2 获取文件名称
p1=$1
fname=`basename $p1`
echo fname=$fname
#3 获取上级目录到绝对路径
pdir=`cd -P $(dirname $p1); pwd`
echo pdir=$pdir
#4 获取当前用户名称
user=`whoami`
#5 循环,这里host根据自己的节点数和主机名设置
for((host=1; host<3; host++)); do
echo --------------- slave$host ----------------
rsync -rvl $pdir/$fname $user@slave$host:$pdir
done
分发flink目录:
xsync /soft/flink
5.启动服务flink集群
#进入目录
cd /soft/flink/bin
#启动集群
./start-cluster.sh
停止集群服务
./stop-cluster.sh
6.测试作业事例
cd /soft/flink/bin
./flink run ../examples/streaming/WordCount.jar
tail ../log/flink-*-taskexecutor-*.out
7.webui地址
http://192.168.189.9:8081/#/overview