服务检查Spring Admin

使用原因

无论是在微服务下服务管理还是单体项目下的服务监控服务宕机的检测其实一直都显得非常重要，毕竟无论是B端的用户还是C端的用户服务不可用是非常糟糕的体验，当然我们可以多节点部署保证服务的高可用，那么即使在多节点部署的情况下我们也不能说不管服务宕机而引起的人肉运维的相关问题，所以SpringAdmin显得就尤为重要了。

简介

• 微服务实践的过程中，我们会发现服务数量众多，如何监测我们哪些服务时正常的、哪些又是亚健康的服务呢？别急，springcloud既然叫微服务全家桶，当然它也有方案啦，我们可以轻松集成springadmin来对我们所有服务的健康状态做监控。

本文将介绍其springadmin如何使用，以及它提供的功能进行说明，内容有

• springadmin都有哪些功能
• 搭建安全的、高可用的springadmin监控服务
• 动态调整日志打印级别
• 配置服务状态变更邮件报警
• 查看服状态，包括CPU、内存、磁盘、缓存、DB、MQ各种服务的连接状态等等
• 查看java虚拟机状态、GC情况和堆栈信息
• 配置监控springboot2.X版本的集群外服务

搭建安全的、高可用的springadmin监控服务

2.1 服务目录结构如下

┍— src/main/java
┊
├----- StartServer.java #服务启动类
┊
├-- src/java/test
┊
├-- src/main/resources
┊
├----- application.yml #可以把这些信息放在配置中心，发布后可以动态修改报警邮箱和规则等
┊
├----- bootstrap.yml #服务的基本信息、启动等信息，可以被application.yml中的配置信息覆盖
┊
┕-- pom.xml

2.2 引入依赖信息

• 编辑pom.xml文件的内容

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>spring-cloud-alibaba</artifactId>
        <groupId>com.snk</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <description>服务监控项目</description>
    <artifactId>snk-admin</artifactId>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!--hutool工具包（难得糊涂）-->
        <dependency>
            <groupId>cn.hutool</groupId>
            <artifactId>hutool-all</artifactId>
        </dependency>
        <!-- springBoot admin 监控 -->
        <dependency>
            <groupId>de.codecentric</groupId>
            <artifactId>spring-boot-admin-starter-server</artifactId>
        </dependency>
        <!--Spring Boot Admin 支持Hazelcast的集群-->
        <dependency>
            <groupId>com.hazelcast</groupId>
            <artifactId>hazelcast</artifactId>
        </dependency>
        <!-- Mysql驱动包 -->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
        </dependency>
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
            <version>2.2.1.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
            <version>2.2.2.RELEASE</version>
        </dependency>
    </dependencies>
    <build>
        <!--       这样也可以把所有的xml文件，打包到相应位置。-->
        <finalName>snk-admin</finalName>
        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <includes>
                    <include>**/*.properties</include>
                    <include>**/*.yml</include>
                    <include>**/*.xml</include>
                    <include>**/*.tld</include>
                </includes>
                <filtering>false</filtering>
            </resource>
            <resource>
                <directory>src/main/java</directory>
                <includes>
                    <include>**/*.properties</include>
                    <include>**/*.yml</include>
                    <include>**/*.xml</include>
                    <include>**/*.tld</include>
                </includes>
                <filtering>false</filtering>
            </resource>
        </resources>

        <!--  这里是保证主类不会找不到。-->
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
            <!-- docker插件 -->
            <plugin>
                <groupId>com.spotify</groupId>
                <artifactId>docker-maven-plugin</artifactId>
                <version>1.0.0</version>
                <configuration>
                    <imageName>${docker.image.prefix}/${project.artifactId}</imageName>
                    <!--docker文件所在的目录-->
                    <dockerDirectory>src/main/docker</dockerDirectory>
                    <resources>
                        <resource>
                            <targetPath>/</targetPath>
                            <directory>${project.build.directory}</directory>
                            <include>${project.build.finalName}.jar</include>
                        </resource>
                    </resources>
                </configuration>
            </plugin>
        </plugins>

    </build>
</project>

2.3 启动类信息

• 编辑StartServer.java

package com.snk.admin;

import com.hazelcast.config.Config;
import com.hazelcast.config.EvictionPolicy;
import com.hazelcast.config.InMemoryFormat;
import com.hazelcast.config.MapConfig;
import com.hazelcast.config.MergePolicyConfig;
import com.hazelcast.map.merge.PutIfAbsentMapMergePolicy;
import de.codecentric.boot.admin.server.config.EnableAdminServer;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;

@SpringBootApplication
@EnableAdminServer
public class StartServer {

    public static void main(String[] args) {
        SpringApplication.run(StartServer.class, args);
    }

    @Bean
    public RestTemplate customRestTemplate() {
        HttpComponentsClientHttpRequestFactory httpRequestFactory = new HttpComponentsClientHttpRequestFactory();
        httpRequestFactory.setConnectionRequestTimeout(3000);
        httpRequestFactory.setConnectTimeout(3000);
        httpRequestFactory.setReadTimeout(3000);
        return new RestTemplate(httpRequestFactory);
    }

    @Bean
    public Config hazelcastConfig() {//集群配置
        MapConfig mapConfig = new MapConfig("spring-boot-admin-event-store").setInMemoryFormat(InMemoryFormat.OBJECT)
                .setBackupCount(1)
                .setEvictionPolicy(EvictionPolicy.NONE)
                .setMergePolicyConfig(new MergePolicyConfig(
                        PutIfAbsentMapMergePolicy.class.getName(),
                        100
                ));
        return new Config().setProperty("hazelcast.jmx", "true").addMapConfig(mapConfig);
    }
}

2.4 配置信息

• 编辑application.yml

server:
  port: 29000


spring:
  boot:
    admin:
      notify:
        web-server:
          enabled: true
  redis:
    host: 127.0.0.1
    port: 6379
    database: 0
    lettuce:
      pool:
        max-active: 8
        max-idle: 8
        max-wait: -1s
        min-idle: 0
    timeout: 100ms
  main:
    ##后定义的覆盖项目中同名的bean
    allow-bean-definition-overriding: true
  datasource:
    type: com.zaxxer.hikari.HikariDataSource
    driver-class-name: com.mysql.cj.jdbc.Driver
    # 主库数据源
    url: jdbc:mysql://127.0.0.1:3306/user-provider?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8
    username: root
    password: 123456
    druid:
      # 初始连接数
      initialSize: 5
      # 最小连接池数量
      minIdle: 10
      # 最大连接池数量
      maxActive: 20
      # 配置获取连接等待超时的时间
      maxWait: 60000
      # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
      timeBetweenEvictionRunsMillis: 60000
      # 配置一个连接在池中最小生存的时间，单位是毫秒
      minEvictableIdleTimeMillis: 300000
      # 配置一个连接在池中最大生存的时间，单位是毫秒
      maxEvictableIdleTimeMillis: 900000
      # 配置检测连接是否有效
      validationQuery: SELECT 1 FROM DUAL
      testWhileIdle: true
      testOnBorrow: false
      testOnReturn: false
      webStatFilter:
        enabled: true
      statViewServlet:
        enabled: true
        # 设置白名单，不填则允许所有访问
        allow:
        url-pattern: /druid/*
        # 控制台管理用户名和密码
        login-username:
        login-password:
      filter:
        stat:
          enabled: true
          # 慢SQL记录
          log-slow-sql: true
          slow-sql-millis: 1000
          merge-sql: true
        wall:
          config:
            multi-statement-allow: true

• 编辑bootstap.yml

##微服务配置
spring:
  cloud:
    nacos:
      config:
        server-addr: 127.0.0.1:8848
        file-extension: yml
        name: snk-admin
        group: DEFAULT_GROUP
        namespace:
      discovery:
        server-addr: 127.0.0.1:8848
        namespace:
  application:
    name: snk-admin
feign:
  hystrix:
    enabled: true
  client:
    config:
      default:
        #Feign的连接建立超时时间，默认为10秒
        connectTimeout: 100000
        #Feign的请求处理超时时间，默认为10秒
        readTimeout: 100000
hystrix:
  command:
    default:
      execution:
        enable: true
        isolation:
          thread:
            timeoutInMilliseconds: 5000
#ribbon的超时时间
ribbon:
  ReadTimeout: 5000 #设置请求处理的超时时间
  ConnectTimeout: 5000  #设置请求链接的超时时间
logging:
  level:
    com.alibaba.nacos.client.config.impl: WARN
    com:
      alibaba:
        nacos:
          client:
            naming: warn

2.5 自定义通知方法

package com.snk.admin.config;

import cn.hutool.json.JSONUtil;
import com.alibaba.fastjson.JSON;
import de.codecentric.boot.admin.server.domain.entities.Instance;
import de.codecentric.boot.admin.server.domain.entities.InstanceRepository;
import de.codecentric.boot.admin.server.domain.events.InstanceEvent;
import de.codecentric.boot.admin.server.domain.events.InstanceStatusChangedEvent;
import de.codecentric.boot.admin.server.notify.AbstractStatusChangeNotifier;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RestTemplate;
import reactor.core.publisher.Mono;

/**
 * @program: admin-server
 * @description: webServer通知
 * @author: tianfn
 * @create: 2021-09-14 17:26
 **/
public class WebServerNotifier extends AbstractStatusChangeNotifier {

    private static final Logger logger = LoggerFactory.getLogger(WebServerNotifier.class);


    @Autowired
    private RestTemplate restTemplate;


    public WebServerNotifier(InstanceRepository repository) {
        super(repository);
    }

   /**
     * 一定要重写doNotify方法，默认将[从UNKNOWN到UP]的通知忽略掉了，导致服务启动识别不到
     * @param event
     * @param instance
     * @return
     */
    @Override
    protected boolean shouldNotify(InstanceEvent event, Instance instance) {
        if (event instanceof InstanceStatusChangedEvent) {
            return true;
        }
        return false;
    }

    @Override
    protected Mono<Void> doNotify(InstanceEvent event, Instance instance) {
        InstanceStatusChangedEvent statusChange = (InstanceStatusChangedEvent) event;
        String from = getLastStatus(event.getInstance());
        String to = statusChange.getStatusInfo().getStatus();
        logger.info("=================== {}服务从{}状态，变为{}： ===================", instance.getRegistration().getName(), from, to);
        return Mono.fromRunnable(() ->
                doHttpRequest(instance));
    }


    public void doHttpRequest(Instance instance) {
        MultiValueMap<String, Object> paramMap = new LinkedMultiValueMap<>();
        paramMap.add("appName", instance.getRegistration().getName());
        paramMap.add("serverId", instance.getId().getValue());
        paramMap.add("serverStats", instance.getStatusInfo().getStatus());

        //TODO instance.getStatusInfo().getStatus()当值为 DOWN/OFFLINE 是执行异步通知服务宕机
        logger.info("检测上线下线通知参数{}", JSON.toJSONString(paramMap));
//        restTemplate.postForObject(url, paramMap, String.class);
    }


    public RestTemplate getRestTemplate() {
        return restTemplate;
    }

    public void setRestTemplate(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }

}

package com.snk.admin.config;

import de.codecentric.boot.admin.server.config.AdminServerNotifierAutoConfiguration;
import de.codecentric.boot.admin.server.domain.entities.InstanceRepository;
import org.springframework.boot.autoconfigure.AutoConfigureBefore;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * @program: admin-server
 * @description: web-server通知配置
 * @author: tianfn
 * @create: 2021-09-14 17:28
 **/
@Configuration
@ConditionalOnProperty(
        prefix = "spring.boot.admin.notify.web-server",
        name = "enabled",
        havingValue = "true"
)
@AutoConfigureBefore({AdminServerNotifierAutoConfiguration.NotifierTriggerConfiguration.class, AdminServerNotifierAutoConfiguration.CompositeNotifierConfiguration.class})
public class WebServerNotifierConfiguration {
    @Bean
    @ConditionalOnMissingBean
    @ConfigurationProperties(prefix = "spring.boot.admin.notify.web-server")
    public WebServerNotifier webServerNotifier(InstanceRepository repository) {
        return new WebServerNotifier(repository);
    }
}

客户端配置

<dependency>
            <groupId>de.codecentric</groupId>
            <artifactId>spring-boot-admin-starter-client</artifactId>
            <version>${spring-admin.version}</version>
        </dependency>

spring:
 application:
   name: service-name
 boot:
   admin: #监控服务相关配置
     client:
       url: http://localhost:1202 #springadminserver地址
       username: admin #spring admin server用户名
       password: 123456 #spring admin server密码
       instance:
         name: ${spring.application.name} #该服务在springadmin中展示的名字配置
         service-url:http://localhost:8282 #此服务暴露给springadmin的地址，用来拉取此服务的健康指标信息
         metadata:
           user.name: admin
           user.password: 123456
           
#监控端点暴露            
management:
 endpoint:
   health:
     show-details: ALWAYS
 endpoints:    
   web:
     exposure:
       include: "*"  

参考文章

• spring-admin官方使用教程
• https://docs.spring.io/spring-boot/docs/
• http://www.danyuanblog.com/blog/app/blog/blogDetail.html?id=5dac7e4c4c636361c5e75e9a