7-2 k8s 示例-使用 StatefulSet 部署 MySQL 一主多从

Sky_Nemo

已于 2023-04-24 15:35:50 修改

阅读量649

点赞数

文章标签： kubernetes mysql

于 2023-04-24 15:27:52 首次发布

本文链接：https://blog.csdn.net/qq_25874461/article/details/130344016

版权

更新时间：2023年4月

参考：运行一个有状态的应用程序 | Kubernetes

参考：MariaDB & K8s: How to replicate MariaDB in K8s - MariaDB.org

部署 MySQL 主从（初版）

Namespace

$ vim ./ns-mysql-demo.yaml
---
apiVersion: v1                                                        
kind: Namespace
metadata:
  name: mysql-demo

应用声明

$ kubectl apply -f ns-mysql-demo.yaml

StorageClass

$ vim sc-nfs.yaml
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: sc-nfs
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner 
# 回收策略此处使用 删除，实验环境方便测试
reclaimPolicy: Delete
mountOptions: 
  - soft
  - nfsvers=4.2
  - noatime     # 访问文件时不更新文件 inode 中的时间戳，高并发环境可提高性能
parameters:
  # 根据 PVC 的namespace 和 PVC 名称来生成路径
  pathPattern: "${.PVC.namespace}/${.PVC.name}"
  archiveOnDelete: "true"

应用声明

$ kubectl apply -f sc-nfs.yaml

Service

$ vim hs-mysql-replica.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: hs-mysql-replica
  namespace: mysql-demo
  labels:
    app: mysql
spec:
  ports:
  - name: server-port
    port: 3306
  clusterIP: None
  selector:
    app.kubernetes.io/name: mysql

应用声明

kubectl apply -f hs-mysql-replica.yaml

Secret

MySQL 的 root 用户密码

# 必须加 -n ，否则会把换行符也用 base64 编码
$ echo -n "qwert123.." | base64
cXdlcnQxMjMuLg==

$ vim secret-mysql-root-auth.yaml
---
apiVersion: v1
kind: Secret
metadata:
  name: secret-mysql-root-auth
  namespace: mysql-demo
type: Opaque
data:
  root-password: cXdlcnQxMjMuLg==

应用声明

$ kubectl apply -f secret-mysql-root-auth.yaml

ConfigMap

$ vim configmap-mysql.yaml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: configmap-mysql
  namespace: mysql-demo
data:
  primary.cnf: |
    [mysqld]
    gtid-mode=ON
    enforce-gtid-consistency
    
  replica.cnf: |
    [mysqld]
    gtid-mode=ON
    enforce-gtid-consistency
    read-only=ON
  
  primary.sql: |
    CREATE USER 'repluser'@'%' IDENTIFIED WITH mysql_native_password BY 'replsecret123..';
    GRANT REPLICATION CLIENT,REPLICATION SLAVE ON *.* TO 'repluser'@'%';
    CREATE DATABASE primary_db;
  
  replica.sql: |
    CHANGE MASTER TO 
    MASTER_HOST='mysql-replica-0.hs-mysql-replica',
    MASTER_USER='repluser',
    MASTER_PASSWORD='replsecret123..',
    MASTER_AUTO_POSITION=1,
    MASTER_CONNECT_RETRY=10;

应用声明

$ kubectl apply -f configmap-mysql.yaml

StatefulSet

镜像选择

使用官方镜像：mysql Tags | Docker Hub

镜像版本：mysql:8.0.32

默认配置文件位置：/etc/my.cnf、/etc/mysql/conf.d/

默认数据存储位置：/var/lib/mysql

其他重要目录：

/docker-entrypoint-initdb.d：放置在该路径的 SQL 语句，将作为初始化语句执行

声明文件

$ vim statefulset-mysql-replica.yaml
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql-replica
  namespace: mysql-demo
  labels:
    app.kubernetes.io/name: mysql
spec:
  serviceName: hs-mysql-replica
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: mysql
  template:
    metadata:
      labels:
        app.kubernetes.io/name: mysql
    spec:
      initContainers:
      - name: init-mysql
        image: mysql:8.0.32
        imagePullPolicy: Always
        command:
        - bash
        - "-c"
        - |
          set -ex
          echo 'Starting init-mysql';
          # Check config map to directory that already exists 
          # (but must be used as a volume for main container)
          ls /mnt/config-map
          # 获取主机名中的 id，以判断是 primary 还是 replica
          [[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1
          ordinal=${
   BASH_REMATCH[1]}
          # 复制对应的配置文件到相应目录
          if [[ $ordinal -eq 0 ]]; then
            # 复制配置文件到配置路径
            cp /mnt/config-map/primary.cnf /etc/mysql/conf.d/server-id.cnf
            # 复制初始化 SQL 语句到对应路径
            cp /mnt/config-map/primary.sql /docker-entrypoint-initdb.d
          else
            cp /mnt/config-map/replica.cnf /etc/mysql/conf.d/server-id.cnf
            cp /mnt/config-map/replica.sql /docker-entrypoint-initdb.d
          fi
          # 给 server-id 添加一个偏移量，防止 server-id=0
          echo server-id=$((3000 + $ordinal)) >> /etc/mysql/conf.d/server-id.cnf
          ls /etc/mysql/conf.d/
          cat /etc/mysql/conf.d/server-id.cnf
          
        volumeMounts:
          - name: configmap-mysql
            mountPath: /mnt/config-map
          - name: initdb
            mountPath: /docker-entrypoint-initdb.d
          - name: mysql-config
            mountPath: /etc/mysql/conf.d/

      restartPolicy: Always
    
      containers:
      - name: mysql
        image: mysql:8.0.32
        env:
        # 设置 root 用户密码
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: secret-mysql-root-auth
              key: root-password
        # 
        - name: MYSQL_INITDB_SKIP_TZINFO
          value: "1"
        # 设置时区
        - name: TZ
          value: "Asia/Shanghai"
        ports:
        - name: server-port
          containerPort: 3306
        # 设置运行 MySQL 的用户，默认 root
        securityContext:
          runAsUser: 65534
          runAsGroup: 65534
          # fsGroup: 3000
        volumeMounts:
        # 挂载配置（初始化容器筛选后的配置）
        - name: mysql-config
          mountPath: /etc/mysql/conf.d/
        - name: initdb
          mountPath: /docker-entrypoint-initdb.d
          
        # 挂载数据卷
        - name: mysql-data
          mountPath: /var/lib/mysql
          
      volumes:
      - name: configmap-mysql
        configMap:
          name: configmap-mysql
      - name: mysql-config
        emptyDir: {
   }
      - name: initdb
        emptyDir: {
   }
          
  volumeClaimTemplates:
  - metadata:
      name: mysql-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: sc-nfs
      resources:
        requests:
          storage: 200Mi

注意：

设置了运行容器的用户 ID 和组，默认为 root。若 NFS 存储侧开启了 root 权限压缩（root_squash）,则会出现 “chown: changing ownership of '/var/lib/mysql/': Operation not permitted” 问题，解决该问题方法有两个：

NFS 存储不压缩 root 权限（no_root_squash）
配置容器的运行用户为非 root 用户（无需与 NFS 侧一致），配置后不可再更改运行用户

检查

检查 Pod 状态

$ kubectl get pods -l app.kubernetes.io/name=mysql -n mysql-demo
NAME              READY   STATUS    RESTARTS   AGE
mysql-replica-0   1/1     Running   0          2m44s
mysql-replica-1   1/1     Running   0          2m38s

检查主从状态

# 登录从节点
$ kubectl exec -it mysql-replica-1 -n mysql-demo bash
# 登录 mysql
bash-4.4$ mysql -u'root' -p'qwert123..'

# 检查从节点同步状态，IO 线程和 SQL 线程均正常运行
mysql> show slave status\G;
......
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
......

部署 MySQL 主从（改进）

上述版本有一个比较大的缺陷，当 MySQL 运行了一段时间后，binlog 进行了切换或删除，此时再扩充 MySQL 实例，会出现主从数据不一致的情况。为了解决这个问题，可以引入备份工具，从实例扩展时，先导入主（primary) 实例实时的备份数据，再开启主从同步

镜像制作

创建 Dockerfile

$ vim Dockerfile
# 一阶段，下载解压 xtrabackup
FROM  rockylinux:9.1 as build

MAINTAINER nemo "sky.nemo@outlook.com"


ADD https://downloads.percona.com/downloads/Percona-XtraBackup-8.0/Percona-XtraBackup-8.0.32-26/binary/tarball/percona-xtrabackup-8.0.32-26-Linux-x86_64.glibc2.17-minimal.tar.gz  /
RUN tar zxf percona-xtrabacku