应用上云-在kubernetes中部署nacos

闲愉

已于 2023-06-30 16:58:21 修改

阅读量906

点赞数

文章标签： kubernetes 云原生服务发现

于 2023-06-30 16:33:30 首次发布

版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文链接：https://blog.csdn.net/weixin_45093078/article/details/131478526

版权

本文将在kubernetes中搭建高可用的nacos集群，官网给出了相关的文档供我们参考：Kubernetes Nacos

根据官网给出的部署nacos的yaml文件可以看出存储采用的都是共享文件存储，即访问方式为ReadWriteMany的存储，可以使用cephfs、NFS等。由于实验环境之前部署了ceph,所以本次实验采用cephfs来提供持久化存储。除此之外还需要部署mysql数据库存储nacos的相关配置信息。

目录

一、部署cephfs存储

二、部署mysql

2.1 编写mysql 的yaml文件

2.2 实验过程

三、导入nacos的sql文件

四、部署nacos

4.1 下载nacos部署包

4.2 修改nacos的yaml

一、部署cephfs存储

使用rook在kubernetes集群中部署ceph

使用ceph作为kubernetes集群的持久化存储（storageclass)

二、部署mysql

部署mysql存储nacos配置文件信息，持久化存储使用cephfs storageclass。

2.1 编写mysql 的yaml文件

apiVersion: v1
kind: Namespace
metadata:
  name: cloud-nacos
---

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  namespace: cloud-nacos
  name: nacos-mysql-pvc
spec:
  storageClassName: rook-cephfs
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
---

apiVersion: apps/v1
kind: StatefulSet
metadata:
  namespace: cloud-nacos
  name: nacos-mysql
spec:
  serviceName: nacos-mysql
  selector:
    matchLabels:
      app: nacos-mysql   
  template:
    metadata:
      labels:
        app: nacos-mysql
    spec:
      containers:
        - image: mysql:5.6
          name: nacos-mysql
          env:
            - name: MYSQL_ROOT_PASSWORD
              value: 1qaz@wsx
          ports:
            - containerPort: 3306
              name: nacos-mysql  
          volumeMounts:
            - name: nacos-mysql-volume
              mountPath: /var/lib/mysql
      volumes:
        - name: nacos-mysql-volume
          persistentVolumeClaim:
            claimName: nacos-mysql-pvc
---

apiVersion: v1
kind: Service
metadata:
  namespace: cloud-nacos
  name: nacos-mysql-svc
spec:
  selector:
    app: nacos-mysql
  type: NodePort
  ports:
    - port: 3306
      targetPort: 3306
      nodePort: 30001

2.2 实验过程

[root@master ~]# kubectl create -f nacos-mysql.yaml 
namespace/cloud-nacos created
persistentvolumeclaim/nacos-mysql-pvc created
statefulset.apps/nacos-mysql created
service/nacos-mysql-svc created

[root@master ~]# kubectl get ns
NAME                   STATUS   AGE
cloud-nacos            Active   100s

[root@master ~]# kubectl get pvc
NAME              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nacos-mysql-pvc   Bound    pvc-6cf49776-cb64-4dc3-aa03-b7073def8bfc   10Gi       RWX            rook-cephfs    105s

[root@master ~]# kubectl get pod -n cloud-nacos
NAME            READY   STATUS    RESTARTS   AGE
nacos-mysql-0   1/1     Running   0          7m50s

[root@master ~]# kubectl get svc -n cloud-nacos
NAME              TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
nacos-mysql-svc   NodePort   10.10.199.231   <none>        3306:30001/TCP   8m4s

[root@master ~]# kubectl exec -it nacos-mysql-0 bash -n cloud-nacos

#可以看到/var/lib/mysql 挂载了一个10G的盘和pvc配置的一致
root@nacos-mysql-0:/# df -h
Filesystem  Size  Used Avail Use% Mounted on
10.10.112.17:6789,10.10.36.234:6789,10.10.39.195:6789:/volumes/csi/csi-vol-1ad0ca06-9d14-4239-86be-ca109a6e949b/432c10ae-6b9b-4309-8e09-7e72dd217955   10G  112M  9.9G   2% /var/lib/mysql


root@nacos-mysql-0:/# mysql -uroot -p1qaz@wsx

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
+--------------------+

三、导入nacos的sql文件

首先需要从官网下载nacos的sql文件：https://github.com/alibaba/nacos/releases/download/2.0.3/nacos-server-2.0.3.tar.gz

连接到数据库后，先创建nacos_config数据库，再导入SQL文件。

#下载nacos-server.tar.gz

[root@master ~]# tar -zvxf nacos-server.tar.gz

[root@master ~]# cd /nacos/conf 
[root@master conf]# ls
nacos-mysql.sql

#将nacos的sql文件拷贝到pod的/目录下
[root@master conf]# kubectl cp nacos-mysql.sql nacos-mysql-0:/   -n cloud-nacos


[root@master conf]# kubectl exec -it nacos-mysql-0 bash -n cloud-nacos
root@nacos-mysql-0:/# ls
nacos-mysql.sql

root@nacos-mysql-0:/# mysql -uroot -p1qaz@wsx

#创建nacos_config数据库
mysql> CREATE DATABASE nacos_config; 

#查看nacos_config的默认字符集为latin1。（实测该字符集可正常导入sql文件，改为utf8mb4导入有报错） 
mysql> SHOW CREATE DATABASE nacos_config;
+--------------+-------------------------------------------------------------------------+
| Database     | Create Database                                                         |
+--------------+-------------------------------------------------------------------------+
| nacos_config | CREATE DATABASE `nacos_config` /*!40100 DEFAULT CHARACTER SET latin1 */ |
+--------------+-------------------------------------------------------------------------+

mysql> USE nacos_config;
Database changed

#导入sql,全部为OK即为导入成功。
mysql> SOURCE /nacos-mysql.sql;
Query OK, 0 rows affected (0.32 sec)

Query OK, 0 rows affected (0.31 sec)

Query OK, 0 rows affected (0.33 sec)

Query OK, 0 rows affected (0.33 sec)

......

#查看数据表
mysql> SHOW TABLES;
+------------------------+
| Tables_in_nacos_config |
+------------------------+
| config_info            |
| config_info_aggr       |
| config_info_beta       |
| config_info_tag        |
| config_tags_relation   |
| group_capacity         |
| his_config_info        |
| permissions            |
| roles                  |
| tenant_capacity        |
| tenant_info            |
| users                  |
+------------------------+

四、部署nacos

4.1 下载nacos部署包

#nacos官网给出了在k8s上部署nacos的方案，下载部署包
git clone https://github.com/nacos-group/nacos-K8s.git

[root@master ~]# cd /root/nacos-K8s/deploy/nacos/

#nacos给出了基于多种持久化存储的nacos部署方案，本次使用的持久化存储是cephfs,因此我们需要在nacos-pvc-ceph.yaml的基础上修改yaml文件。
[root@master nacos]# ls
nacos-no-pvc-ingress.yaml  nacos-pvc-ceph.yaml  nacos-pvc-nfs.yaml  nacos-quick-start.yaml

#nacos-pvc-ceph.yaml文件中共创建了三个k8s资源对象，分别是service、ConfigMap、StatefulSet。

4.2 修改nacos的yaml

---
apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  namespace: cloud-nacos             # 增加命名空间
  labels:
    app: nacos
  annotations:
    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
  ports:
    - port: 8848
      name: server
      targetPort: 8848
    - port: 9848
      name: client-rpc
      targetPort: 9848
    - port: 9849
      name: raft-rpc
      targetPort: 9849
    - port: 7848
      name: old-raft-rpc
      targetPort: 7848
  clusterIP: None
  selector:
    app: nacos
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
  namespace: cloud-nacos                             # 增加命名空间
data:
  mysql.host: "nacos-mysql-svc.cloud-nacos"          # 增加mysql连接地址，集群内部只要通过DNS访问，格式为：mysql的service名称.namespace的名称
  mysql.db.name: "nacos_config"                      # 数据库名称
  mysql.port: "3306"                                 # mysql端口，默认3306
  mysql.user: "root"                                 # mysql用户名
  mysql.password: "1qaz@wsx"                         # mysql用户密码
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: cloud-nacos                             # 增加命名空间
spec:
  serviceName: nacos-headless
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos
              topologyKey: "kubernetes.io/hostname"
      initContainers:
        - name: peer-finder-plugin-install
          image: nacos/nacos-peer-finder-plugin:1.1
          imagePullPolicy: Always
          volumeMounts:
            - mountPath: /home/nacos/plugins/peer-finder
              name: nacos-data
              subPath: peer-finder
      containers:
        - name: nacos
          imagePullPolicy: Always
          image: nacos/nacos-server:2.0.3  # 修改nacos版本
          resources:
            requests:
              memory: "1500Mi"
              cpu: "200Mi"
          ports:
            - containerPort: 8848
              name: client-port
            - containerPort: 9848
              name: client-rpc
            - containerPort: 9849
              name: raft-rpc
            - containerPort: 7848
              name: old-raft-rpc
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: SERVICE_NAME
              value: "nacos-headless"
            - name: DOMAIN_NAME
              value: "cluster.local"
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: MYSQL_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.host
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.db.name
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.port
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.user
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.password
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: NACOS_APPLICATION_PORT
              value: "8848"

            - name: PREFER_HOST_MODE
              value: "hostname"
          volumeMounts:
            - name: nacos-data
              mountPath: /home/nacos/plugins/peer-finder
              subPath: peer-finder
            - name: nacos-data
              mountPath: /home/nacos/data
              subPath: data
            - name: nacos-data
              mountPath: /home/nacos/logs
              subPath: logs
  volumeClaimTemplates:
    - metadata:
        name: nacos-data
        namespace: cloud-nacos   # 增加命名空间
      spec:
        accessModes: [ "ReadWriteMany" ]
        storageClassName: rook-cephfs  # 修改存储类名称
        resources:
          requests:
            storage: 5Gi
  selector:
    matchLabels:
      app: nacos


#创建一个nodeport类型的service，用于访问nacos网页

apiVersion: v1
kind: Service
metadata:
  name: nacos-server  
  namespace: cloud-nacos               # 增加命名空间
  labels:
    app: nacos
  annotations:
    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
  type: NodePort
  ports:
    - port: 8848
      name: server
      targetPort: 8848
      nodePort: 30848
  selector:
    app: nacos



kubectl create -f  nacos-pvc-ceph.yaml
#创建完成后访问nacos,网址:http://172.31.246.31:30848/nacos;账号密码：nacos/nacos

五、测试

测试：

#cluster-ip是指service的cluster-ip,对于本集群就是10.10.113.2

[root@master ~]# kubectl get svc -n cloud-nacos
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
nacos-headless    ClusterIP   None            <none>        8848/TCP,9848/TCP,9849/TCP,7848/TCP   24h
nacos-mysql-svc   NodePort    10.10.199.231   <none>        3306:30001/TCP                        29h
nacos-server      NodePort    10.10.113.2     <none>        8848:30848/TCP                        24h


服务注册：
curl -X POST 'http://cluster-ip:8848/nacos/v1/ns/instance?serviceName=nacos.naming.serviceName&ip=20.18.7.10&port=8080'

服务发现：
curl -X GET 'http://cluster-ip:8848/nacos/v1/ns/instance/list?serviceName=nacos.naming.serviceName'

发布配置：
curl -X POST "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test&content=helloWorld"

获取配置：
curl -X GET “http://127.0.0.1:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test”



[root@master ~]# curl -X POST 'http://10.10.113.2:8848/nacos/v1/ns/instance?serviceName=nacos.naming.serviceName&ip=20.18.7.10&port=8080'
ok   

[root@master ~]# curl -X GET 'http://10.10.113.2:8848/nacos/v1/ns/instance/list?serviceName=nacos.naming.serviceName'
{"name":"DEFAULT_GROUP@@nacos.naming.serviceName","groupName":"DEFAULT_GROUP","clusters":"","cacheMillis":10000,"hosts":[],"lastRefTime":1687855146785,"checksum":"","allIPs":false,"reachProtectionThreshold":false,"valid":true} 

[root@master ~]# curl -X POST "http://10.10.113.2:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test&content=helloWorld"
true

[root@master ~]# curl -X GET "http://10.10.113.2:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test"
helloWorld

关注

0
点赞
踩
4

收藏

觉得还不错? 一键收藏
0
评论
应用上云-在kubernetes中部署nacos

在kubernetes中搭建高可用的nacos集群。
复制链接

扫一扫

评论

被折叠的条评论为什么被折叠?

到【灌水乐园】发言

查看更多评论

添加红包

成就一亿技术人!

hope_wisdom

发出的红包

实付元

使用余额支付

点击重新获取

扫码支付

钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。