flink on k8s(Flink1.15.0 on Native Kubernetes)

实验环境

macOS x86-64

minikube

准备工作

安装minikube

下载安装

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64
sudo install minikube-darwin-amd64 /usr/local/bin/minikube
minikube start --driver=hyperkit
或
minikube start

minikube dashboard

minikube dashboard

新建namespace并开启RBAC

# 设置专属的namespace--flinkjob
kubectl create namespace flinkjob

# 创建一个新的 flink服务账户(flink)
kubectl create serviceaccount flink -n flinkjob

#  设置角色绑定
kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit --serviceaccount=flinkjob:flink

Application模式

构建镜像

FROM flink
RUN mkdir -p $FLINK_HOME/usrlib
COPY /path/of/my-flink-job.jar $FLINK_HOME/usrlib/my-flink-job.jar

或者测试直接用./examples/streaming/TopSpeedWindowing.jar，可以使用flink1.15.0基础镜像

加载镜像到集群内

minikube客户端将直接与集群中的容器运行时对话，并在那里对同一存储运行负载命令。

$ minikube image load my_image

向k8s提交作业

./bin/flink run-application \
    --target kubernetes-application \
    -Dkubernetes.cluster-id=my-first-application-cluster \
    -Dkubernetes.container.image=custom-image-name \
    -Dkubernetes.namespace=flinkjob \
    local:///opt/flink/usrlib/my-flink-job.jar

./bin/flink run-application \
-target kubernetes-application \
-Dkubernetes.cluster-id=my-test-flink-k8s \
-Dkubernetes.container.image=flinkk8s:1.15.0 \
-Dkubernetes.rest-service.exposed.type=NodePort \
-Dkubernetes.jobmanager.cpu=0.2 \
-Dkubernetes.taskmanager.cpu=0.1 \
-Dtaskmanager.numberOfTaskSlots=1 \ 
-Djobmanager.memory.process.size=1024m \
-Dtaskmanager.memory.process.size=1024m \
-Dkubernetes.namespace=flinkjob \
local:///opt/flink/examples/streaming/TopSpeedWindowing.jar

./bin/flink run-application -p 1 -t kubernetes-application \
  --detached \
  --allowNonRestoredState \			# 从指定savepoint启动，不需要时可省略掉
  --fromSavepoint oss://flinkjob-state-prod/savepoint-f1e2d7-3108369d7445 \		# 从指定savepoint启动不需要时可省略掉
  -Dkubernetes.namespace=flinkjob \
  -Dkubernetes.jobmanager.service-account=flink \
  -Dkubernetes.cluster-id=flink-jobs \
  -Dkubernetes.container.image.pull-policy=Always \
  -Dkubernetes.container.image=*****/flink-jobs:git.c2f4bba9ffbc26a077ae73d8508fcce0e05752ed \		# 对应镜像地址
  -Djobmanager.memory.process.size=1024m \
  -Dtaskmanager.memory.process.size=1024m \
  -Dkubernetes.jobmanager.cpu=0.2 \
  -Dkubernetes.taskmanager.cpu=0.1 \
  -Dtaskmanager.numberOfTaskSlots=1 \
  -Dkubernetes.rest-service.exposed.type=NodePort \
 local:///flink/flink-jobs/flink-jobs.jar		# 镜像需要执行的jar地址

与k8s集群交互

# 查看当前集群任务id
./bin/flink list --target kubernetes-application -Dkubernetes.cluster-id=flink-jobs -Dkubernetes.namespace=flinkjob

# stop 任务并进行状态savepoint
./bin/flink stop --savepointPath --target kubernetes-application -Dkubernetes.cluster-id=flink-production -Dkubernetes.namespace=flinkjob <job-id>

# stop集
echo 'stop' | ./bin/kubernetes-session.sh -Dkubernetes.cluster-id=flink-production -Dkubernetes.namespace=flinkjob -Dexecution.attached=true

# List running job on the cluster
$ ./bin/flink list --target kubernetes-application -Dkubernetes.cluster-id=my-first-application-cluster
# Cancel running job
$ ./bin/flink cancel --target kubernetes-application -Dkubernetes.cluster-id=my-first-application-cluster <jobId>

访问WebUI

Flink的Web UI和REST端点可以通过kubernetes.rest-service.exposed.type配置选项以多种方式公开。

• ClusterIP：在集群内部IP上公开服务。该服务只能在集群内访问。如果您想访问JobManager UI或将作业提交到现有会话，您需要启动本地代理。然后，您可以使用localhost:8081向会话提交Flink作业或查看仪表板。

$ kubectl port-forward service/<ServiceName> 8081 

• NodePort：在静态端口（NodePort）上公开每个节点IP上的服务。<NodeIP>:<NodePort>可用于联系JobManager服务。
• LoadBalancer: Exposes the service externally using a cloud provider’s load balancer. Since the cloud provider and Kubernetes needs some time to prepare the load balancer, you may get a NodePort JobManager Web Interface in the client log. You can use kubectl get services/<cluster-id>-rest to get EXTERNAL-IP and construct the load balancer JobManager Web Interface manually http://<EXTERNAL-IP>:8081.

访问webUI：http://localhost:8081

---

附：RBAC​​​​​​​

Role-based access control (RBAC) is a method of regulating access to compute or network resources based on the roles of individual users within an enterprise. Users can configure RBAC roles and service accounts used by JobManager to access the Kubernetes API server within the Kubernetes cluster.

Every namespace has a default service account. However, the default service account may not have the permission to create or delete pods within the Kubernetes cluster. Users may need to update the permission of the default service account or specify another service account that has the right role bound.

$ kubectl create clusterrolebinding flink-role-binding-default --clusterrole=edit --serviceaccount=default:default 

If you do not want to use the default service account, use the following command to create a new flink-service-account service account and set the role binding. Then use the config option -Dkubernetes.service-account=flink-service-account to make the JobManager pod use the flink-service-account service account to create/delete TaskManager pods and leader ConfigMaps. Also this will allow the TaskManager to watch leader ConfigMaps to retrieve the address of JobManager and ResourceManager.

$ kubectl create serviceaccount flink-service-account $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit --serviceaccount=default:flink-service-account 

Please refer to the official Kubernetes documentation on RBAC Authorization for more information.