flink部署模式（一）- standalone和application

这里采用docker方式部署实现flink的standalone和application部署模式，由于local模式是standalone模式的特例，所以这里就不再演示。flink版本flink1.15.0。

docker制作及上传私有镜像

Dockfile

tee Dockerfile <<EOF
FROM docker.io/flink:java8
RUN mkdir -p $FLINK_HOME/usrlib
COPY flink-demo-1.0-SNAPSHOT-pony-shade.jar $FLINK_HOME/usrlib/flink-demo-1.0-SNAPSHOT-pony-shade.jar
EOF

镜像构建

docker build -t ponylee/flink:1.15.0-java8 .

登录本地镜像仓库：

docker login -u username -p passwd

推送镜像：

docker push ponylee/flink:1.15.0-java8

测试镜像可用性

docker run --name flink -d ponylee/flink:1.15.0-java8 jobmanager
docker exec -it flink bash

Application Mode on Docker

创建网络

docker network create flink-network-app

开启jobmanager

docker run -d \
--name flink-jm \
--hostname flink-jm \
-p 8082:8081 \
--env FLINK_PROPERTIES="jobmanager.rpc.address: flink-jm" \
--network flink-network-app \
ponylee/flink:1.15.0-java8 \
standalone-job --job-classname com.pony.mock.TopSpeedWindowing 

args: [“standalone-job”, “–job-classname”, “com.job.ClassName”, , ]
可选的参数项: [“–job-id”, “”, “–fromSavepoint”, “/path/to/savepoint”, “–allowNonRestoredState”]

开启taskmanager

docker run -d \
--name flink-tm \
--hostname flink-tm \
--env FLINK_PROPERTIES="jobmanager.rpc.address: flink-jm" \
--network flink-network-app \
ponylee/flink:1.15.0-java8 \
taskmanager \
-Dtaskmanager.numberOfTaskSlots=10 

Standalone Mode on Docker

创建网络

docker network create flink-network-standalone

开启jobmanager

docker run -d \
--name flink-jm \
--hostname flink-jm \
-p 8082:8081 \
--env FLINK_PROPERTIES="jobmanager.rpc.address: flink-jm" \
--network flink-network-standalone \
ponylee/flink:1.15.0-java8  \
jobmanager 

开启taskmanager

docker run -d \
--name flink-tm \
--hostname flink-tm \
--env FLINK_PROPERTIES="jobmanager.rpc.address: flink-jm" \
--network flink-network-standalone \
ponylee/flink:1.15.0-java8 \
taskmanager \
-Dtaskmanager.memory.process.size=1024m \
-Dtaskmanager.numberOfTaskSlots=5 \
-Drest.flamegraph.enabled=true

提交任务到standalone集群

docker exec -it flink-jm bin/flink run examples/streaming/TopSpeedWindowing.jar