statefulset部署etcd v3.5.4集群

本文目的是使用statefulset部署3节点的etcd集群

一。准备双AZ 4节点的集群

构建集群参考另一篇帖子CentOS7安装k8s v1.23.4集群

k8s-node1,k8s-node2打上标签topology.kubernetes.io/zone=az1,表示AZ1;

k8s-node3,k8s-node4打上标签topology.kubernetes.io/zone=az2,表示AZ2;

 二。构建etcd v3.5.4镜像

自行构建docker镜像,从etcd官网下载etcd-v3.5.4-linux-amd64.tar.gz并解压,

 Dockerfile配置:

FROM centos:7

USER root

ADD etcd-v3.5.4-linux-amd64/etcd* /usr/bin/

COPY etcd.sh /

RUN chmod 777 /etcd.sh

RUN chmod 777 /usr/bin/etcd

CMD ["/etcd.sh"]

容器启动脚本etcd.sh,环境变量由pod注入,见statefulset中的配置

#!/bin/bash
echo ${MY_POD_NAME}
echo ${MY_POD_IP}
echo ${HEADLESS_SERVICE_NAME}
echo ${CLUSTER_NAMESPACE}
echo ${INITIAL_CLUSTER}

/usr/bin/etcd --data-dir=/var/lib/etcd --name=${MY_POD_NAME} --listen-peer-urls=http://0.0.0.0:2380 --listen-client-urls=http://0.0.0.0:2379 --advertise-client-urls=http://${MY_POD_NAME}.${HEADLESS_SERVICE_NAME}.${CLUSTER_NAMESPACE}.svc.cluster.local:2379 --initial-advertise-peer-urls=http://${MY_POD_NAME}.${HEADLESS_SERVICE_NAME}.${CLUSTER_NAMESPACE}.svc.cluster.local:2380 --initial-cluster-state='new' --initial-cluster-token='etcd-cluster-token' --initial-cluster=${INITIAL_CLUSTER}

 构建镜像并推送到阿里个人镜像仓库

也可直接docker pull registry.cn-shenzhen.aliyuncs.com/k8s_go_java/etcd:v1

三。使用local-storage存储类提供pv

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

创建pv前,先在每个节点创建pv对应的目录/mnt/statefulset/etcd,注意pv配置了节点亲和

apiVersion: v1
kind: PersistentVolume
metadata:
  name: etcd-pv
spec:
  capacity:
    storage: 2Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /mnt/statefulset/etcd
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: topology.kubernetes.io/zone
          operator: In
          values:
          - az1


---

apiVersion: v1
kind: PersistentVolume
metadata:
  name: etcd-pv-2
spec:
  capacity:
    storage: 2Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /mnt/statefulset/etcd
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: topology.kubernetes.io/zone
          operator: In
          values:
          - az1


---

apiVersion: v1
kind: PersistentVolume
metadata:
  name: etcd-pv-3
spec:
  capacity:
    storage: 2Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /mnt/statefulset/etcd
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: topology.kubernetes.io/zone
          operator: In
          values:
          - az2


---

apiVersion: v1
kind: PersistentVolume
metadata:
  name: etcd-pv-4
spec:
  capacity:
    storage: 2Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /mnt/statefulset/etcd
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: topology.kubernetes.io/zone
          operator: In
          values:
          - az2

 四。创建svc

一个名为etcd-headless-svc的无头服务,一个NodePort类型的svc

apiVersion: v1
kind: Service
metadata:
  name: etcd-headless-svc
  namespace: etcd
  labels:
    app: etcd      
spec:
  clusterIP: None
  ports:
  - name: etcd-client
    port: 2379
    targetPort: 2379
    protocol: TCP
  - name: etcd-peer
    port: 2380
    targetPort: 2380
    protocol: TCP
  selector:
    app: etcd


---


apiVersion: v1
kind: Service
metadata:
  name: etcd-nodeport
  namespace: etcd
  labels:
    app: etcd
spec:
  type: NodePort
  ports:
  - name: etcd-cluster
    port: 2379
    targetPort: 2379
    protocol: TCP
  sessionAffinity: None
  selector:
    app: etcd

五。创建statefulset

这里创建了两个sts:etcd-az1和etcd-az2,目的是限制sts中的pod跨AZ调度,即etcd-az1中的pod只能在AZ1下的k8s-node1和k8s-node2中调度,而不能在AZ2的节点中调度,保障AZ级别的高可用,注意nodeSelector中对AZ的配置。

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: etcd-az1
  namespace: etcd
  labels:
    app.kubernetes.io/name: etcd-cluster-az1
    app: etcd
spec:
  podManagementPolicy: Parallel
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: etcd-cluster-az1
  serviceName: etcd-headless-svc
  template:
    metadata:
      labels:
        app: etcd
        app.kubernetes.io/name: etcd-cluster-az1
    spec:
      nodeSelector:
        topology.kubernetes.io/zone: az1
      containers:
      - name: etcd
        image: registry.cn-shenzhen.aliyuncs.com/k8s_go_java/etcd:v1
        imagePullPolicy: IfNotPresent
        ports:
        - name: client
          containerPort: 2379
          protocol: TCP
        - name: peer
          containerPort: 2380
          protocol: TCP
        env:
        - name: MY_POD_NAME  #当前pod名
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: MY_POD_IP  #当前pod ip
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: CLUSTER_NAMESPACE  #名称空间
          valueFrom: 
            fieldRef:
              fieldPath: metadata.namespace
        - name: HEADLESS_SERVICE_NAME   #内部通信的无头服务名称
          value: "etcd-headless-svc"
        - name: INITIAL_CLUSTER   #initial-cluster的值
          value: "etcd-az1-0=http://etcd-az1-0.etcd-headless-svc.etcd.svc.cluster.local:2380,etcd-az1-1=http://etcd-az1-1.etcd-headless-svc.etcd.svc.cluster.local:2380,etcd-az2-0=http://etcd-az2-0.etcd-headless-svc.etcd.svc.cluster.local:2380"
        resources:
          requests:
            cpu: "1000m"
            memory: "1Gi"
          limits:
            cpu: "1000m"
            memory: "1Gi"
        volumeMounts:
        - name: data
          mountPath: /var/lib/etcd
  updateStrategy:
    type: OnDelete
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: local-storage
      resources:
        requests:
          storage: 2Gi
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: etcd-az2
  namespace: etcd
  labels:
    app.kubernetes.io/name: etcd-cluster-az2
    app: etcd
spec:
  podManagementPolicy: Parallel
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: etcd-cluster-az2
  serviceName: etcd-headless-svc
  template:
    metadata:
      labels:
        app: etcd
        app.kubernetes.io/name: etcd-cluster-az2
    spec:
      nodeSelector:
        topology.kubernetes.io/zone: az2
      containers:
      - name: etcd
        image: registry.cn-shenzhen.aliyuncs.com/k8s_go_java/etcd:v1
        imagePullPolicy: IfNotPresent
        ports:
        - name: client
          containerPort: 2379
          protocol: TCP
        - name: peer
          containerPort: 2380
          protocol: TCP
        env:
        - name: MY_POD_NAME  #当前pod名
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: MY_POD_IP  #当前pod ip
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: CLUSTER_NAMESPACE  #名称空间
          valueFrom: 
            fieldRef:
              fieldPath: metadata.namespace
        - name: HEADLESS_SERVICE_NAME   #内部通信的无头服务名称
          value: "etcd-headless-svc"
        - name: INITIAL_CLUSTER   #initial-cluster的值
          value: "etcd-az1-0=http://etcd-az1-0.etcd-headless-svc.etcd.svc.cluster.local:2380,etcd-az1-1=http://etcd-az1-1.etcd-headless-svc.etcd.svc.cluster.local:2380,etcd-az2-0=http://etcd-az2-0.etcd-headless-svc.etcd.svc.cluster.local:2380"
        resources:
          requests:
            cpu: "1000m"
            memory: "1Gi"
          limits:
            cpu: "1000m"
            memory: "1Gi"
        volumeMounts:
        - name: data
          mountPath: /var/lib/etcd
  updateStrategy:
    type: OnDelete
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: local-storage
      resources:
        requests:
          storage: 2Gi

六。验证

用NodePort方式访问etcd集群

 

  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值