k8s安装mongodb副本集

1 创建命名空间

# mongodb-ns.yaml
# 为mongodb创建命名空间
apiVersion: v1
kind: Namespace
metadata:
  name: mongodb-ns

2 创建NFS和StorageClass

注意：在“192.168.108.100”上，安装NFS，并创建共享目录“/data/mongodb”。

# mongodb-nfs-storage.yaml

# mongodb-nfs-client-provisioner-authority.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: mongodb-ns
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: mongodb-ns
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: mongodb-ns
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: mongodb-ns
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: mongodb-ns
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

---
# mongodb-nfs-client-provisioner.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  namespace: mongodb-ns
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            # 存储分配器名称
            - name: PROVISIONER_NAME
              value: mongodb-nfs-provisioner
            # NFS服务器地址，设置为自己的IP
            - name: NFS_SERVER
              value: 192.168.108.100
            # NFS共享目录地址
            - name: NFS_PATH
              value: /data/mongodb
      volumes:
        - name: nfs-client-root
          nfs:
            # 设置为自己的IP
            server: 192.168.108.100
            # 对应NFS上的共享目录
            path: /data/mongodb

---
# mongodb-nfs-storage-class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mongodb-nfs-storage
  namespace: mongodb-ns

# 存储分配器的名称
# 对应“mongodb-nfs-client-provisioner.yaml”文件中env.PROVISIONER_NAME.value
provisioner: mongodb-nfs-provisioner

# 允许pvc创建后扩容
allowVolumeExpansion: True

parameters:
  # 资源删除策略，“true”表示删除PVC时，同时删除绑定的PV
  archiveOnDelete: "true"

3 设置权限

# mongodb-authority.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: mongodb
  namespace: mongodb-ns
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: mongo-default-view
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
  - kind: ServiceAccount
    name: mongodb
    namespace: mongodb-ns

4 创建MongoDB服务

# mongodb-service.yaml
# Headless Service维护MongoDB实例之间的集群关系，及集群规模变化的自动更新
apiVersion: v1
kind: Service
metadata:
  name: mongodb-in-svc
  namespace: mongodb-ns
spec:
  clusterIP: None
  # 注意，此处标签名称含有mongodb-cluster.yaml中的selector.matchLabels标签
  # 服务（Headless Service）通过此属性找控制器
  selector:
    app: mongodb
  ports:
    - name: mongodb-in-svc-port
      port: 27017
      targetPort: 27017
---
# 向外暴漏MongoDB端口
apiVersion: v1
kind: Service
metadata: 
  name: mongodb-out-svc
  namespace: mongodb-ns
spec:
  type: NodePort
  selector: 
    app: mongodb
  ports: 
  - name: mongodb-out-svc-port
    # 服务端口
    port: 27017
    # 节点对应的端口
    nodePort: 30017

5 创建MongoDB控制器

# mongodb-cluster.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongodb-cluster
  namespace: mongodb-ns
spec:
  selector:
    # 注意：要与mongodb-service.yaml文件中NodePort Service中的selector相同
    matchLabels: 
      app: mongodb
  # 注意，要与mongodb-service.yaml文件中Headless Service中的metadata.name相同
  # 控制器（StatefulSet）通过此属性找服务（Headless Service）
  serviceName: mongodb-in-svc
  replicas: 3
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: mongodb
      containers:
        - name: mongodb-node
          image: mongo:4.2.8
          imagePullPolicy: IfNotPresent
          command:
            - mongod
            - "--port"
            - "27017"
            - "--shardsvr"
            - "--replSet"
            - rs0
            - "--bind_ip"
            - 0.0.0.0
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongodb-data
              mountPath: /data/db
            
  volumeClaimTemplates:
    - metadata:
        name: mongodb-data
      spec:
        accessModes: [ "ReadWriteMany" ]
        storageClassName: mongodb-nfs-storage
        resources:
          requests:
            storage: 10Gi

查看MongoDB的相关信息

# 查看Pod
kubectl get pod -n mongodb-ns
# 返回值
NAME                                      READY   STATUS    RESTARTS   AGE
mongodb-cluster-0                         1/1     Running   0          21s
mongodb-cluster-1                         1/1     Running   0          18s
mongodb-cluster-2                         1/1     Running   0          13s
nfs-client-provisioner-548d9496dc-8rmvg   1/1     Running   0          10h

# 查看service
kubectl get svc -n mongodb-ns
# 返回值
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)           AGE
mongodb-in-svc    ClusterIP   None            <none>        27017/TCP         10h
mongodb-out-svc   NodePort    10.11.168.165   <none>        27017:30017/TCP   10h

6 初始化副本集群

6.1 查看集群状态

# 进入Pod
kubectl exec -it mongodb-cluster-0 /bin/bash -n mongodb-ns

# 查看Headless Service是否成功连接MongoDB集群
# 使用DNS访问的格式{ServiceName}.{NameSpace}.svc.{ClusterDomain}
# 此处：ServiceName=mongodb-in-svc，NameSpace=mongodb-ns，ClusterDomain=cluster.local
# 27017是每个节点的端口号
mongo mongodb-cluster-0.mongodb-in-svc.mongodb-ns.svc.cluster.local:27017

# 连接成功后会出现如下结果
MongoDB shell version v4.2.8
connecting to: mongodb://mongodb-in-svc.mongodb-ns.svc.cluster.local:27017/test?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("446ee0fb-859b-4f4f-8466-4ed797fb2e9b") }
MongoDB server version: 4.2.8
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
	http://docs.mongodb.org/
Questions? Try the support group
	http://groups.google.com/group/mongodb-user
Server has startup warnings: 
2022-05-10T12:18:48.346+0000 I  CONTROL  [initandlisten] 
2022-05-10T12:18:48.346+0000 I  CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2022-05-10T12:18:48.347+0000 I  CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2022-05-10T12:18:48.347+0000 I  CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2022-05-10T12:18:48.347+0000 I  CONTROL  [initandlisten] 
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---

> 

6.2 初始化副本集

# （1）将配置信息放入shell中执行
cfg = {
    _id : "rs0",
    members : [
    { _id:0, host:"mongodb-cluster-0.mongodb-in-svc.mongodb-ns.svc.cluster.local:27017" },
    { _id:1, host:"mongodb-cluster-1.mongodb-in-svc.mongodb-ns.svc.cluster.local:27017" },
    { _id:2, host:"mongodb-cluster-2.mongodb-in-svc.mongodb-ns.svc.cluster.local:27017" }
    ]
}


# （2）初始化集群
rs.initiate(cfg)

# 如下返回值，表示成功
{
	"ok" : 1,
	"$clusterTime" : {
		"clusterTime" : Timestamp(1652189347, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	},
	"operationTime" : Timestamp(1652189347, 1)
}