k8s集群搭建redis集群

k8s集群搭建redis集群

注： 有状态、可持续换集成
redis集群有两种部署方式
StatefulSet
Service&Deployment

一、创建NFS存储

找一个k8s节点安装（我是在master节点安装）

1. 创建NFS存储
   创建NFS存储主要是为了给Redis提供稳定的后端存储，当Redis的Pod发生故障重启或迁移后，依然能获得原先的数据。
   这里，我们先要创建NFS，然后通过使用PV为Redis挂载一个远程的NFS路径。
2. 安装NFS

yum -y install nfs-utils （提供文件系统）
yum -y install rpcbind	（提供rpc协议）

3. 创建共享文件

mkdir -p /usr/local/k8s/redis/pv{1..6}

vim /etc/exports

#添加共享路径，增加权限
/usr/local/k8s/redis/pv1 192.168.2.0/24(rw,sync,no_root_squash)
/usr/local/k8s/redis/pv2 192.168.2.0/24(rw,sync,no_root_squash)
/usr/local/k8s/redis/pv3 192.168.2.0/24(rw,sync,no_root_squash)
/usr/local/k8s/redis/pv4 192.168.2.0/24(rw,sync,no_root_squash)
/usr/local/k8s/redis/pv5 192.168.2.0/24(rw,sync,no_root_squash)
/usr/local/k8s/redis/pv6 192.168.2.0/24(rw,sync,no_root_squash)

4. 重启服务

systemctl restart rpcbind
systemctl restart nfs
systemctl enable nfs

5. 查看exportfs

exportfs  -v

6. 在其他的节点安装nfs-utils客户端（我是在其他的node节点安装）

yum -y install nfs-utils

7. 在其他及诶单查看存储共享

showmount -e 192.168.2.228

二、创建PV

vim pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv1
spec:
  capacity:
    storage: 300M
  accessModes:
    - ReadWriteMany
  nfs:
    server: 192.168.2.228
    path: "/usr/local/k8s/redis/pv1"

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-vp2
spec:
  capacity:
    storage: 300M
  accessModes:
    - ReadWriteMany
  nfs:
    server: 192.168.2.228
    path: "/usr/local/k8s/redis/pv2"

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv3
spec:
  capacity:
    storage: 300M
  accessModes:
    - ReadWriteMany
  nfs:
    server: 192.168.2.228
    path: "/usr/local/k8s/redis/pv3"

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv4
spec:
  capacity:
    storage: 300M
  accessModes:
    - ReadWriteMany
  nfs:
    server: 192.168.2.228
    path: "/usr/local/k8s/redis/pv4"

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv5
spec:
  capacity:
    storage: 300M
  accessModes:
    - ReadWriteMany
  nfs:
    server: 192.168.2.228
    path: "/usr/local/k8s/redis/pv5"

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv6
spec:
  capacity:
    storage: 300M
  accessModes:
    - ReadWriteMany
  nfs:
    server: 192.168.2.228
    path: "/usr/local/k8s/redis/pv6"

创建pv

kubectl create -f pv.yaml 

三、Configmap

vim redis.conf

appendonly yes
cluster-enabled yes
cluster-config-file /var/lib/redis/nodes.conf
cluster-node-timeout 5000
dir /var/lib/redis
port 6379

创建Configmap

kubectl create configmap redis-conf --from-file=redis.conf

查看创建的configmap

kubectl describe cm redis-conf

四、创建Redis 集群节点

vim redis.yaml

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: redis-app
spec:
  serviceName: "redis-service"
  replicas: 6
  template:
    metadata:
      labels:
        app: redis
        appCluster: redis-cluster
    spec:
      terminationGracePeriodSeconds: 20
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - redis
              topologyKey: kubernetes.io/hostname
      containers:
      - name: redis
        image: redis
        command: 
          [ "/bin/bash", "-ce","redis-server /etc/redis/redis.conf" ]
        args:
          - "/etc/redis/redis.conf"
          - "--protected-mode"
          - "no"
        resources:
          requests:
            cpu: "100m"
            memory: "100Mi"
        ports:
            - name: redis
              containerPort: 6379
              protocol: "TCP"
            - name: cluster
              containerPort: 16379
              protocol: "TCP"
        volumeMounts:
          - name: "redis-conf"
            mountPath: "/etc/redis"
          - name: "redis-data"
            mountPath: "/var/lib/redis"
      volumes:
      - name: "redis-conf"
        configMap:
          name: "redis-conf"
          items:
            - key: "redis.conf"
              path: "redis.conf"
  volumeClaimTemplates:
  - metadata:
      name: redis-data
    spec:
      accessModes: [ "ReadWriteMany" ]
      resources:
        requests:
          storage: 100M

创建redis.yaml节点

kubectl create -f redis.yaml

查看redis的pod节点（不要着急，pod是一个一个启动的）

kubectl get pods -o wide

五、创建集群

kubectl get pods -o wide （获取所有的IP）
kubectl exec -it redis-app-0 /bin/bash
redis-cli --cluster create --cluster-replicas  1 10.244.2.66:6379 10.244.4.58:6379 10.244.3.47:6379 10.244.4.59:6379 10.244.2.67:6379 10.244.3.48:6379
yes
#成功之后
redis-cli -h 10.244.2.66 -p 6379
cluster info
cluster nodes

六、创建Headless service

vim headless-service.yaml 

apiVersion: v1
kind: Service
metadata:
  name: redis-service
  labels:
    app: redis
spec:
  type: NodePort
  ports:
  - name: redis-port
    port: 6379
  selector:
    app: redis
    appCluster: redis-cluster

创建service

kubectl create -f headless-service.yml

kubectl get svc

也可以根据service的ip+port连接redis集群

可能中间还会出现问题，请谅解，谢谢大家支持！！！！