微服务系列：Nacos的搭建演武版（单机+集群+监控）

一、前言

Docker版：Docker初识：Nacos服务的搭建（单机版+集群版）

官网版：Nacos官方文档

二、安装说明：

系统：centos 7

mysql：8

nacos：2.0.3

安装包下载地址：https://github.com/alibaba/Nacos/releases

官方安装文档：Nacos支持三种部署模式

• 单机模式 - 用于测试和单机试用。
• 集群模式 - 用于生产环境，确保高可用。
• 多集群模式 - 用于多数据中心场景。

三、单机模式

单机模式很简单，可以直接参照官网安装即可。

下载安装包。

解压并进入目录。

[root@VM_0_6_centos nacos]# wget https://github.com/alibaba/nacos/releases/download/2.0.3/nacos-server-2.0.3.tar.gz
[root@VM_0_6_centos nacos]# ls
nacos-server-2.0.3.tar.gz 
[root@VM_0_6_centos nacos]# tar -zxvf nacos-server-2.0.3.tar.gz
nacos/LICENSE
nacos/NOTICE
nacos/target/nacos-server.jar
nacos/conf/
nacos/conf/1.4.0-ipv6_support-update.sql
nacos/conf/schema.sql
nacos/conf/nacos-mysql.sql
nacos/conf/application.properties.example
nacos/conf/nacos-logback.xml
nacos/conf/cluster.conf.example
nacos/conf/application.properties
nacos/bin/startup.sh
nacos/bin/startup.cmd
nacos/bin/shutdown.sh
nacos/bin/shutdown.cmd
[root@VM_0_6_centos nacos]# ls
nacos  nacos-server-2.0.3.tar.gz 
[root@VM_0_6_centos nacos]# cd nacos/
[root@VM_0_6_centos nacos]# ll
total 36
drwxr-xr-x 2 root root   4096 Oct 28 09:51 bin
drwxr-xr-x 2  502 games  4096 Jul 27 14:18 conf
-rw-r--r-- 1  502 games 16583 Mar 18  2021 LICENSE
-rw-r--r-- 1  502 games  1305 May 14  2020 NOTICE
drwxr-xr-x 2 root root   4096 Oct 28 09:51 target
[root@VM_0_6_centos nacos]# cd bin/
[root@VM_0_6_centos bin]# ls
shutdown.cmd  shutdown.sh  startup.cmd  startup.sh

解压后，直接运行即可。

bin/startup.sh -m standalone

[root@VM_0_6_centos nacos]# bin/startup.sh -m standalone
/opt/jdk1.8.0_171/bin/java -Djava.ext.dirs=/opt/jdk1.8.0_171/jre/lib/ext:/opt/jdk1.8.0_171/lib/ext  -Xms512m -Xmx512m -Xmn256m -Dnacos.standalone=true -Dnacos.member.list= -Xloggc:/opt/nacos/nacos/logs/nacos_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dloader.path=/opt/nacos/nacos/plugins/health,/opt/nacos/nacos/plugins/cmdb -Dnacos.home=/opt/nacos/nacos -jar /opt/nacos/nacos/target/nacos-server.jar  --spring.config.additional-location=file:/opt/nacos/nacos/conf/ --logging.config=/opt/nacos/nacos/conf/nacos-logback.xml --server.max-http-header-size=524288
nacos is starting with standalone
nacos is starting，you can check the /opt/nacos/nacos/logs/start.out
[root@VM_0_6_centos nacos]# tail -f logs/start.out
2021-10-28 10:08:35,776 INFO Creating filter chain: any request, [org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@140d1230, org.springframework.security.web.context.SecurityContextPersistenceFilter@6c6366cf, org.springframework.security.web.header.HeaderWriterFilter@1daf3b44, org.springframework.security.web.csrf.CsrfFilter@1a6cf771, org.springframework.security.web.authentication.logout.LogoutFilter@481e91b6, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@18acfe88, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@6631cb64, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@3cbf1ba4, org.springframework.security.web.session.SessionManagementFilter@44c13103, org.springframework.security.web.access.ExceptionTranslationFilter@53ed09e8]

2021-10-28 10:08:35,937 INFO Initializing ExecutorService 'taskScheduler'

2021-10-28 10:08:35,973 INFO Exposing 16 endpoint(s) beneath base path '/actuator'

2021-10-28 10:08:36,103 INFO Tomcat started on port(s): 8848 (http) with context path '/nacos'

2021-10-28 10:08:36,107 INFO Nacos started successfully in stand alone mode. use embedded storage

访问nocas的管理端：http://IP:8848/nacos ，默认的用户名密码是 nocas/nocas

当然，除了使用嵌入式数据库还可以使用外置数据源。以下摘自官网。

在0.7版本之前，在单机模式时nacos使用嵌入式数据库实现数据的存储，不方便观察数据存储的基本情况。0.7版本增加了支持mysql数据源能力，具体的操作步骤：

• 1.安装数据库，版本要求：5.6.5+
• 2.初始化mysql数据库，数据库初始化文件：nacos-mysql.sql
• 3.修改conf/application.properties文件，增加支持mysql数据源配置（目前只支持mysql），添加mysql数据源的url、用户名和密码。

spring.datasource.platform=mysql

db.num=1
db.url.0=jdbc:mysql://11.162.196.16:3306/nacos_devtest?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=nacos_devtest
db.password=youdontknow

注：mysql数据库,sql文件位置：conf\nacos-mysql.sql

四、集群模式

1、单机搭建伪集群，复制nacos安装包，修改为nacos8849，nacos8850，nacos8851

[root@VM_0_6_centos nacos]# ls
nacos  nacos8849  nacos8859  nacos8869  

2、以nacos8849为例，进入nacos8849目录。

1）修改conf\application.properties的配置，使用外置数据源。

 在0.7版本之前，在单机模式时nacos使用嵌入式数据库实现数据的存储，不方便观察数据存储的基本情况。0.7版本增加了支持mysql数据源能力，具体的操作步骤：

• 1.安装数据库，版本要求：5.6.5+
• 2.初始化mysql数据库，数据库初始化文件：nacos-mysql.sql
• 3.修改conf/application.properties文件，增加支持mysql数据源配置（目前只支持mysql），添加mysql数据源的url、用户名和密码。

spring.datasource.platform=mysql

db.num=1
db.url.0=jdbc:mysql://11.162.196.16:3306/nacos_devtest?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=nacos_devtest
db.password=youdontknow

2）修改将conf\cluster.conf.example改为cluster.conf,添加节点配置。

[root@VM_0_6_centos conf]# cat cluster.conf
#
# Copyright 1999-2018 Alibaba Group Holding Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

#it is ip
#example
#192.168.16.101:8847
IP1:8859
IP2:8869
IP3:8849

3）根据需要修改JVM参数或者不修改。

4）nacos8859，nacos8869 按同样的方式配置。

3、分别启动nacos8849，nacos8859，nacos8869即可。

访问nocas的管理端：http://IP:8849/nacos ，用户名密码是 nocas/nocas，查看其他两个端点。

官方推荐nginx反向代理：

stream {

    upstream nacosCluster {
        server IP1:8849;
		server IP2:8859;
		server IP3:8869;
    }

    server {
        listen       8850;
        server_name  localhost;

        #charset koi8-r;

        access_log  logs/host.access.log;

		location /nacos/ {
            proxy_pass http://nacosCluster/nacos/;
        }
		
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

}

访问地址：http://IP:8850/nacos ，用户名密码是 nocas/nocas，查看其他两个端点。

到此完毕。

五、prometheus+grafana监控Nacos

官方文档：Nacos 监控手册

Nacos 0.8.0版本完善了监控系统，支持通过暴露metrics数据接入第三方监控系统监控Nacos运行状态，目前支持prometheus、elastic search和influxdb，下面结合prometheus和grafana如何监控Nacos，官网grafana监控页面。与elastic search和influxdb结合可自己查找相关资料

1、搭建prometheus采集Nacos metrics数据

分别修改nacos8849，nacos8859，nacos8869的conf/application.properties，暴露metrics数据。

#*************** Metrics Related Configurations ***************#
### Metrics for prometheus
management.endpoints.web.exposure.include=*

重启后，测试一下，看看是否正确。

http://IP:8849/nacos/actuator/prometheus

2、下载安装prometheus。

下载地址：Download | Prometheus

解压后并修改配置文件prometheus.yml采集Nacos metrics数据。

protometheus.yml四个部分：

• global:  全局配置（如果有内部单独设定，会覆盖这个参数）
• alerting: 告警插件定义。这里会设定alertmanager这个报警插件。
• rule_files: 告警规则。 按照设定参数进行扫描加载，用于自定义报警规则，其报警媒介和route路由由alertmanager插件实现。
• scrape_configs:采集配置。配置数据源，包含分组job_name以及具体target。又分为静态配置和服务发现

具体参考：Prometheus配置文件prometheus.yml 四个模块详解

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    metrics_path: '/nacos/actuator/prometheus'
    static_configs:
      - targets: ['10.32.2.17:8849','10.32.2.17:8859','10.32.2.17:8869']

启动prometheus服务。

#默认9090端口
./prometheus --config.file="prometheus.yml" &
#修改为指定启动端口
./prometheus --config.file="prometheus.yml" --web.listen-address=:8840 &

访问：http://IP:8840/graph，搜索栏搜索nacos_monitor可以搜索到Nacos数据说明采集数据成功。

 3、搭建grafana图形化展示metrics数据

使用yum安装。

[root@VM_0_6_centos nacos88]$ sudo yum install https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-5.2.4-1.x86_64.rpm
已加载插件：fastestmirror, langpacks
grafana-5.2.4-1.x86_64.rpm                                                                                                                                                                |  52 MB  00:12:31     
正在检查 /var/tmp/yum-root-mQIJKH/grafana-5.2.4-1.x86_64.rpm: grafana-5.2.4-1.x86_64
/var/tmp/yum-root-mQIJKH/grafana-5.2.4-1.x86_64.rpm 将被安装
正在解决依赖关系
--> 正在检查事务
---> 软件包 grafana.x86_64.0.5.2.4-1 将被 安装
--> 解决依赖关系完成
base/7/x86_64                                                                                                                                                                             | 3.6 kB  00:00:00     
base/7/x86_64/primary_db                                                                                                                                                                  | 6.1 MB  00:00:01     
docker-ce-stable/x86_64                                                                                                                                                                   | 3.5 kB  00:00:00     
docker-ce-stable/x86_64/primary_db                                                                                                                                                        |  67 kB  00:00:00     
extras/7/x86_64                                                                                                                                                                           | 2.9 kB  00:00:00     
extras/7/x86_64/primary_db                                                                                                                                                                | 243 kB  00:00:00     
updates/7/x86_64                                                                                                                                                                          | 2.9 kB  00:00:00     
updates/7/x86_64/primary_db                                                                                                                                                               |  12 MB  00:00:10     

依赖关系解决

=================================================================================================================================================================================================================
 Package                                       架构                                         版本                                             源                                                             大小
=================================================================================================================================================================================================================
正在安装:
 grafana                                       x86_64                                       5.2.4-1                                          /grafana-5.2.4-1.x86_64                                       158 M

事务概要
=================================================================================================================================================================================================================
安装  1 软件包

总计：158 M
安装大小：158 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  正在安装    : grafana-5.2.4-1.x86_64                                                                                                                                                                       1/1 
### NOT starting on installation, please execute the following statements to configure grafana to start automatically using systemd
 sudo /bin/systemctl daemon-reload
 sudo /bin/systemctl enable grafana-server.service
### You can start grafana-server by executing
 sudo /bin/systemctl start grafana-server.service
POSTTRANS: Running script
  验证中      : grafana-5.2.4-1.x86_64                                                                                                                                                                       1/1 

已安装:
  grafana.x86_64 0:5.2.4-1                                                                                                                                                                                       

完毕！
[root@VM_0_6_centos nacos88]$sudo service grafana-server start
Starting grafana-server (via systemctl):  [  确定  ]

注意：

[root@VM_0_6_centos nacos88]$ service grafana-server start
Starting grafana-server (via systemctl):  ==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to manage system services or units.
Authenticating as: home
Password: 
==== AUTHENTICATION FAILED ===
Failed to start grafana-server.service: Access denied
See system logs and 'systemctl status grafana-server.service' for details.
                                                           [失败]

访问地址：http://IP:3000/，默认账号/密码：admin/admin.

官方使用说明：With Grafana | Grafana Labs

进入主页后，配置prometheus数据源。

 其他的根据需要填写。

导入Nacos grafana监控模版

至此，演武完毕，其他细节则需一一测试，不在此赘述。

六、遇到的问题

1、三个节点，其中正常启动两个，一个启动报错：org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'grpcClusterServer': Invocation of init method failed; nested exception is java.io.IOException: Failed to bind

具体错误如下：

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'grpcClusterServer': Invocation of init method failed; nested exception is java.io.IOException: Failed to bind
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:139)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:413)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1761)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:592)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:514)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:321)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:319)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:866)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
        at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1204)
        at com.alibaba.nacos.Nacos.main(Nacos.java:35)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467)
Caused by: java.io.IOException: Failed to bind
        at io.grpc.netty.shaded.io.grpc.netty.NettyServer.start(NettyServer.java:246)
        at io.grpc.internal.ServerImpl.start(ServerImpl.java:184)
        at io.grpc.internal.ServerImpl.start(ServerImpl.java:90)
        at com.alibaba.nacos.core.remote.grpc.BaseGrpcServer.startServer(BaseGrpcServer.java:152)
        at com.alibaba.nacos.core.remote.BaseRpcServer.start(BaseRpcServer.java:47)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:363)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:307)
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136)
        ... 26 common frames omitted
Caused by: io.grpc.netty.shaded.io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: Address already in use
2021-10-28 16:44:57,831 WARN [WatchFileCenter] start close

2021-10-28 16:44:57,831 WARN [WatchFileCenter] start to shutdown this watcher which is watch : /opt/nacos/nacos8851/conf

2021-10-28 16:44:57,831 WARN [WatchFileCenter] start to shutdown this watcher which is watch : /opt/nacos/nacos8851/data/loader

2021-10-28 16:44:57,831 WARN [WatchFileCenter] start to shutdown this watcher which is watch : /opt/nacos/nacos8851/data/tps

2021-10-28 16:44:57,831 WARN [WatchFileCenter] already closed

2021-10-28 16:44:57,831 WARN [NotifyCenter] Start destroying Publisher

2021-10-28 16:44:57,831 WARN [NotifyCenter] Destruction of the end

2021-10-28 16:44:57,831 ERROR Nacos failed to start, please see /opt/nacos/nacos8851/logs/nacos.log for more details.

^C

前因后果：

最初三个节点的端口分别是8849、8850、8851，但是在启动过程中发现总是一个节点无法启动，并提示上述错误，经排查发现并非是端口占用导致的。从网上搜索的半天，终于发现一个博客说明了原因，为此我特意到官方网站去看了下，找到了答案。

 

 上述问题解决参考博客：

Nacos2.0.1数据持久化和伪集群学习--集群启动报错问题（Error creating bean with name ‘grpcSdkServer‘）

 找到了问题所在，把三个节点端口换成8849、8859、8869，经测试可以正常启动。

2、nacos正常启动，但是浏览器无法访问。

如果服务器是正常的，这就得看下你的防火墙了。