apollo集群部署_在K8S中部署Nacos配置中心

点击上方 蓝字 关注我们

服务注册和配置中心是运维工作中经常遇见的字眼，配置中心要实现的一个基本理念是配置修改的时效性、动态性、一致性；服务注册和发现是为了解耦服务之间的依赖关系和便于服务管理。

为什么需要配置中心

随着业务的发展、微服务架构的升级，服务的数量、程序的配置日益增多(各种微服务、各种服务器地址、各种参数)，传统的配置文件方式和数据库的方式已无法满足开发人员对配置管理的要求：

• 安全性：配置跟随源代码保存在代码库中，容易造成配置泄漏。

• 时效性：修改配置，需要重启服务才能生效。

• 局限性：无法支持动态调整：例如日志开关、功能开关。

因此，分布式配置中心应运而生！

工作中常用的配置中心：

1. Spring Cloud Config：2014年9月开源，Spring Cloud 生态组件，可以和Spring Cloud体系无缝整合。2. Apollo：2016年5月，携程开源的配置管理中心，具备规范的权限、流程治理等特性，很多知名公司在用。3. Nacos：2018年6月，阿里开源的配置中心，也可以做DNS和RPC的服务发现。

对于Spring Cloud Config，在此不再多说，运维工作中主流的配置中心还是apollo和nacos

我们为什么选择nacos

1. 在权限管理这块apollo有其独特优势，nacos在权限管理这块尚需优化中；2. Nacos部署简化，Nacos整合了注册中心、配置中心功能，且部署相比apollo简单，方便管理和监控；3. apollo容器化较困难，Nacos有官网的镜像可以直接部署，总体来说，Nacos比apollo更符合KISS原则；4. 性能方面，Nacos读写tps比apollo稍强一些

k8s集群信息

worker01   Ready    controlplane,etcd,master,worker   50d   v1.18.3worker02   Ready    controlplane,etcd,worker          50d   v1.18.3worker03   Ready    controlplane,etcd,worker          50d   v1.18.3worker04   Ready    worker                            50d   v1.18.3worker05   Ready    worker                            50d   v1.18.3worker06   Ready    worker                            50d   v1.18.3worker07   Ready    worker                            37d   v1.18.3worker08   Ready    worker                            37d   v1.18.3worker09   Ready    worker                            37d   v1.18.3# 其中worker07--09作为此次nacos集群的部署节点，设有label标签app=true

部署mysql数据库

代码克隆

git clone https://github.com/nacos-group/nacos-k8s.git

数据库pvc创建 (使用已有ceph)

[root@worker01 deploy]# kubectl get scNAME                 PROVISIONER          RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGExsky-rbd (default)   iscsi.csi.xsky.com   Delete          Immediate           true                   50d[root@worker01 deploy]# cat ceph/pvc.yaml ---apiVersion: v1kind: PersistentVolumeClaimmetadata:  name: mysql-nacos-pvcspec:  accessModes:    - ReadWriteOnce  storageClassName: xsky-rbd  resources:    requests:      storage: 10Gi [root@worker01 deploy]# kubectl create ns test1     [root@worker01 deploy]# kubectl apply -f ceph/pvc.yaml -n test1

mysql副本(单节点)和服务创建，数据库部署目录nacos-k8s/deploy/mysql

[root@worker01 mysql]# cat mysql-ceph.yaml apiVersion: v1kind: ReplicationControllermetadata:  name: nacos-mysql  labels:    name: nacos-mysqlspec:  replicas: 1  selector:    name: nacos-mysql  template:    metadata:      labels:        name: nacos-mysql    spec:      containers:      - name: nacos-mysql        image: nacos/nacos-mysql:5.7        ports:        - containerPort: 3306        env:        - name: MYSQL_ROOT_PASSWORD          value: "root"        - name: MYSQL_DATABASE          value: "nacos"        - name: MYSQL_USER          value: "nacos"        - name: MYSQL_PASSWORD          value: "nacos"        volumeMounts:        - name: mysql-nacos          mountPath: /var/lib/mysql          readOnly: false      volumes:      - name: mysql-nacos        persistentVolumeClaim:          claimName: mysql-nacos-pvc---apiVersion: v1kind: Servicemetadata:  name: mysql-nacos  labels:    name: mysql-nacosspec:  ports:  - port: 3306    targetPort: 3306  selector:    name: nacos-mysql

部署nacos

目录nacos-k8s/deploy/nacos，修改nacos-pvc-ceph.yaml，主要修改configmap和sts的NACOS_SERVERS和数据持久化(新增application.properties配置文件的持久化)

[root@worker01 nacos]# cat application.properties                                                    # springserver.servlet.contextPath=${SERVER_SERVLET_CONTEXTPATH:/nacos}server.contextPath=/nacosserver.port=${NACOS_APPLICATION_PORT:8848}spring.datasource.platform=${SPRING_DATASOURCE_PLATFORM:""}nacos.cmdb.dumpTaskInterval=3600nacos.cmdb.eventTaskInterval=10nacos.cmdb.labelTaskInterval=300nacos.cmdb.loadDataAtStart=falsedb.num=${MYSQL_DATABASE_NUM:1}db.url.0=jdbc:mysql://${MYSQL_SERVICE_HOST}:${MYSQL_SERVICE_PORT:3306}/${MYSQL_SERVICE_DB_NAME}?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=truedb.url.1=jdbc:mysql://${MYSQL_SERVICE_HOST}:${MYSQL_SERVICE_PORT:3306}/${MYSQL_SERVICE_DB_NAME}?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=truedb.user=${MYSQL_SERVICE_USER}db.password=${MYSQL_SERVICE_PASSWORD}### The auth system to use, currently only 'nacos' is supported:nacos.core.auth.system.type=${NACOS_AUTH_SYSTEM_TYPE:nacos}### The token expiration in seconds:nacos.core.auth.default.token.expire.seconds=${NACOS_AUTH_TOKEN_EXPIRE_SECONDS:18000}### The default token:nacos.core.auth.default.token.secret.key=${NACOS_AUTH_TOKEN:SecretKey012345678901234567890123456789012345678901234567890123456789}### Turn on/off caching of auth information. By turning on this switch, the update of auth information would have a 15 seconds delay.nacos.core.auth.caching.enabled=${NACOS_AUTH_CACHE_ENABLE:false}server.tomcat.accesslog.enabled=${TOMCAT_ACCESSLOG_ENABLED:false}server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D# default current work dirserver.tomcat.basedir=## spring security config### turn off securitynacos.security.ignore.urls=/,/error,/**/*.css,/**/*.js,/**/*.html,/**/*.map,/**/*.svg,/**/*.png,/**/*.ico,/console-fe/public/**,/v1/auth/**,/v1/console/health/**,/actuator/**,/v1/console/server/**# metrics for elastic searchmanagement.metrics.export.elastic.enabled=falsemanagement.metrics.export.influx.enabled=falsenacos.naming.distro.taskDispatchThreadCount=10nacos.naming.distro.taskDispatchPeriod=200nacos.naming.distro.batchSyncKeyCount=1000nacos.naming.distro.initDataRatio=0.9nacos.naming.distro.syncRetryDelay=5000nacos.naming.data.warmup=true

[root@worker01 nacos]# kubectl create -n test1 cm nacos-config --from-file=application.properties

[root@worker01 nacos]# cat nacos-pvc-ceph.yaml ---apiVersion: v1kind: Servicemetadata:  name: nacos-headless  labels:    app: nacos  annotations:    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"spec:  clusterIP: None  ports:    - port: 8848      name: server      targetPort: 8848      protocol: TCP  selector:    app: nacos---apiVersion: v1kind: ConfigMapmetadata:  name: nacos-cmdata:  mysql.db.name: "nacos"  mysql.port: "3306"  mysql.user: "nacos"  mysql.password: "nacos"  mysql.service.host: "mysql-nacos.test1.svc.cluster.local" #test1为我的namespace---apiVersion: apps/v1kind: StatefulSetmetadata:  name: nacosspec:  serviceName: nacos-headless  replicas: 3  template:    metadata:      labels:        app: nacos      annotations:        pod.alpha.kubernetes.io/initialized: "true"    spec:      affinity:        podAntiAffinity:          requiredDuringSchedulingIgnoredDuringExecution:            - labelSelector:                matchExpressions:                  - key: "app"                    operator: In                    values:                      - 'true'              topologyKey: "kubernetes.io/hostname"#      serviceAccountName: block-csi-provisioner      initContainers:        - name: peer-finder-plugin-install          image: nacos/nacos-peer-finder-plugin:1.0          imagePullPolicy: IfNotPresent          volumeMounts:            - mountPath: /home/nacos/plugins/peer-finder              name: plugindir      containers:        - name: nacos          imagePullPolicy: IfNotPresent          image: nacos/nacos-server:1.3.2          resources:            requests:              memory: "2Gi"              cpu: "500m"          ports:            - containerPort: 8848              name: client-port          env:            - name: NACOS_REPLICAS              value: "2"            - name: SERVICE_NAME              value: "nacos-headless"            - name: DOMAIN_NAME              value: "cluster.local"            - name: POD_NAMESPACE              valueFrom:                fieldRef:                  apiVersion: v1                  fieldPath: metadata.namespace            - name: MYSQL_SERVICE_HOST              valueFrom:                configMapKeyRef:                  name: nacos-cm                  key: mysql.service.host            - name: MYSQL_SERVICE_DB_NAME              valueFrom:                configMapKeyRef:                  name: nacos-cm                  key: mysql.db.name            - name: MYSQL_SERVICE_PORT              valueFrom:                configMapKeyRef:                  name: nacos-cm                  key: mysql.port            - name: MYSQL_SERVICE_USER              valueFrom:                configMapKeyRef:                  name: nacos-cm                  key: mysql.user            - name: MYSQL_SERVICE_PASSWORD              valueFrom:                configMapKeyRef:                  name: nacos-cm                  key: mysql.password            - name: NACOS_SERVER_PORT              value: "8848"            - name: NACOS_APPLICATION_PORT              value: "8848"            - name: PREFER_HOST_MODE              value: "hostname"            - name: NACOS_SERVERS              value: "nacos-0.nacos-headless.test1.svc.cluster.local:8848 nacos-1.nacos-headless.test1.svc.cluster.local:8848 nacos-2.nacos-headless.test1.svc.cluster.local:8848"          volumeMounts:            - name: plugindir              mountPath: /home/nacos/plugins/peer-finder            - name: datadir              mountPath: /home/nacos/data            - name: logdir              mountPath: /home/nacos/logs            - name: nacos-config              mountPath: /home/nacos/conf/application.properties              subPath: application.properties      volumes:      - name: nacos-config        configMap:          name: nacos-config     volumeClaimTemplates:    - metadata:        name: plugindir      spec:        accessModes: [ "ReadWriteOnce" ]        storageClassName: "xsky-rbd"        resources:          requests:            storage: 1Gi    - metadata:        name: datadir      spec:        accessModes: [ "ReadWriteOnce" ]        storageClassName: "xsky-rbd"        resources:          requests:            storage: 6Gi    - metadata:        name: logdir      spec:        accessModes: [ "ReadWriteOnce" ]        storageClassName: "xsky-rbd"        resources:          requests:            storage: 5Gi  selector:    matchLabels:      app: nacos

[root@worker01 nacos]# kubectl apply -f nacos-pvc-ceph.yaml -n test1       service/nacos-headless createdconfigmap/nacos-cm createdstatefulset.apps/nacos created

Ingress服务暴露

apiVersion: extensions/v1beta1kind: Ingressmetadata:  name: nacos-ingress  namespace: test1spec:  rules:  - host: nacos.domain.test    http:      paths:      - backend:          serviceName: nacos-headless          servicePort: 8848        path: /nacos

浏览器访问http://nacos.domain.test/nacos

出其东门

 

出其东门，有女如云。

虽则如云。匪我思存。

缟衣綦巾，聊乐我员。

出其闉阇，有女如荼。

虽则如荼，匪我思且。

缟衣茹藘，聊可与娱。

求关注

求转发