kubernetes部署dolphinscheduler

kubernetes部署dolphinscheduler+mysql

1. 下载所需镜像

docker pull apache/dolphinscheduler-master:3.0.0-alpha
docker pull apache/dolphinscheduler-worker:3.0.0-alpha
docker pull apache/dolphinscheduler-alert-server:3.0.0-alpha
docker pull apache/dolphinscheduler-api:3.0.0-alpha
docker pull apache/dolphinscheduler-tools:3.0.0-alpha

2. 使用mysql作为dolphinscheduler数据库

1. 下载mysql驱动器https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar
2. 基于官方镜像重新制作master、api、alert-server、tools镜像

FROM apache/dolphinscheduler-${组件名}:3.0.0-alpha
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/libs
``
```shell
docker build -t apache/dolphinscheduler-${组件名}:mysql-driver .

3. worker组件容器镜像需要修改applicatiom.yaml文件

spring:
  banner:
    charset: UTF-8
  application:
    name: worker-server
  jackson:
    time-zone: UTC
    date-format: "yyyy-MM-dd HH:mm:ss"
  datasource:
    #driver-class-name: org.postgresql.Driver
    driver-class-name: com.mysql.cj.jdbc.Driver
    #url: jdbc:postgresql://127.0.0.1:5432/dolphinscheduler
    url: jdbc:mysql://mysql服务ip:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false&serverTimezone=UTC&autoReconnect=true
    username: root
    password: root
    port: 3306
    database: "dolphinscheduler"
registry:
  type: zookeeper
  zookeeper:
    namespace: dolphinscheduler
    connect-string: localhost:2181
    retry-policy:
      base-sleep-time: 60ms
      max-sleep: 300ms
      max-retries: 5
    session-timeout: 30s
    connection-timeout: 9s
    block-until-connected: 600ms
    digest: ~
worker:
  # worker listener port
  listen-port: 1234
  # worker execute thread number to limit task instances in parallel
  exec-threads: 100
  # worker heartbeat interval, the unit is second
  heartbeat-interval: 10
  # worker host weight to dispatch tasks, default value 100
  host-weight: 100
  # worker tenant auto create
  tenant-auto-create: true
  # worker max cpuload avg, only higher than the system cpu load average, worker server can be dispatched tasks. default value -1: the number of cpu cores * 2
  max-cpu-load-avg: -1
  # worker reserved memory, only lower than system available memory, worker server can be dispatched tasks. default value 0.3, the unit is G
  reserved-memory: 0.3
  # default worker groups separated by comma, like 'worker.groups=default,test'
  groups:
    - default
  # alert server listen host
  alert-listen-host: localhost
  alert-listen-port: 50052
server:
  port: 1235
management:
  endpoints:
    web:
      exposure:
        include: '*'
  metrics:
    tags:
      application: ${spring.application.name}
metrics:
  enabled: true

4. 制作worker镜像

FROM apache/dolphinscheduler-worker:3.0.0-alpha
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/libs
COPY application.yaml /opt/dolphinscheduler/application.yaml

docker build -t apache/dolphinscheduler-worker:mysql-driver .

3. 下载dolphinscheduler相关版本文件

wget https://dlcdn.apache.org/dolphinscheduler/3.0.0-alpha/apache-dolphinscheduler-3.0.0-alpha-src.tar.gz
tar -zxvf apache-dolphinscheduler-3.0.0-alpha-src.tar.gz
cd apache-dolphinscheduler-3.0.0-alpha-src/docker/kubernetes/dolphinscheduler

4. 修改values.yaml文件

1. 修改 values.yaml 文件中 image 的 repository 字段，并更新 tag 为 mysql-driver

   image:
     registry: "apache"
     tag: "mysql-driver"
   ......
   ``
   

2. 修改 values.yaml 文件中 postgresql 的 enabled 为 false

   postgresql:
     enabled: false
   ......
   

3. 修改 values.yaml 文件中的 externalDatabase 配置 (尤其修改 host, username 和 password)

   externalDatabase:
     type: "mysql"
     driver: "com.mysql.cj.jdbc.Driver"
     host: "mysql-data-svc.infra"
     port: "3306"
     username: "root"
     password: "MWz87jpt6GpN@trhis"
     database: "dolphinscheduler"
     params: "useUnicode=true&characterEncoding=UTF-8&useSSL=false&serverTimezone=UTC&autoReconnect=true"
   

4. 修改service类型为NodePort

   ......
   service:
       type: "NodePort"
   ......
   

5. 数据持久化将value.yaml文件中相应组件下的persistentVolumeClaim 值改为true，有以下两种pvc创建方式

   persistentVolumeClaim:
       enabled: true
   

   （1）提前准备各组件需要的pvc，修改templates/各个组件的yaml文件，volumes修改为创建好的pvc，例如：

   volumes:
     - name: {{ include "dolphinscheduler.fullname" . }}-alert
   {{- if .Values.alert.persistentVolumeClaim.enabled }}
       persistentVolumeClaim:
       claimName: pvc-alter-server  #修改为自己的pvc名称      
   

   （2）修改storageClassName自动创建pvc

   persistentVolumeClaim:
       ......
         storageClassName: "-"
       ......
   

5. 准备postgresql和zoopeer，如果不需要创建postgresql和zoopeer可省略

cd apache-dolphinscheduler-3.0.0-alpha-src/docker/kubernetes/dolphinscheduler
helm repo add bitnami https://charts.bitnami.com/bitnami
helm dependency update .

5.1 如果需要安装postgresql和zoopeer其中一个

（1）修改Chart.lock文件，例如

dependencies:
#- name: postgresql
#  repository: https://charts.bitnami.com/bitnami
#  version: 10.3.18
- name: zookeeper
  repository: https://charts.bitnami.com/bitnami
  version: 6.5.3

（2）修改Chart.yaml文件，例如

dependencies:
#- name: postgresql
#  version: 10.3.18
#  repository: https://charts.bitnami.com/bitnami
#  condition: postgresql.enabled
- name: zookeeper
  version: 6.5.3
  repository: https://charts.bitnami.com/bitnami
  condition: zookeeper.enabled

6. 开始安装dolphinscheduler

cd apache-dolphinscheduler-3.0.0-alpha-src/docker/kubernetes/dolphinscheduler
helm install dolphinscheduler . -n 命名空间
#卸载
#helm uninstall dolphinscheduler  -n 命名空间