RabbitMQ 连接异常导致容器中的SpringBoot 项目退出问题分析

1、首先描述一下问题：

      今日在开发环境提测时，发现了部署在容器中的后台服务一直在重新启动，导致无法提供服务。

2、项目背景

    采用了 Springboot+nacos，消息中心使用mq，使用依赖

    // rabbitmq
    implementation 'org.springframework.boot:spring-boot-starter-amqp:2.1.7.RELEASE'

3、分析定位问题

   1）步骤重现，无特定步骤使用平台几分钟后便无法提供访问。

   2）查看k8s Pod 详情，使用kubectl describe pod ，发现pod 健康状态为Unhealthy，导致无法对外提供服务。

   3）查看deployment的健康检查机制

          livenessProbe:
            httpGet:
              scheme: HTTP
              path: /actuator/health
              port: 20016
            initialDelaySeconds: 120
            periodSeconds: 15

      使用了springboot 框架的actuator 健康检查

   4）再查看pod的日志信息

2020-11-19 20:23:12 | INFO  | grpc-default-executor-7 | org.springframework.amqp.rabbit.connection.CachingConnectionFactory | Attempting to connect to: [192.168.**.***]
2020-11-19 20:23:12 | ERROR | grpc-default-executor-7 | com.***i.mq.impl.PlatformMqServiceImpl | 发送消息失败，Exception：
org.springframework.amqp.AmqpIOException: java.io.IOException
	at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71)
	at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:530)
	at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:702)
	at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.createConnection(ConnectionFactoryUtils.java:214)
	at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2076)
	at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2050)
	at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2030)
	at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueProperties(RabbitAdmin.java:403)
...
	at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:172)
	at org.apache.skywalking.apm.plugin.grpc.v1.CallServerInterceptor$ServerCallListener.onHalfClose(CallServerInterceptor.java:144)
	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
	at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:817)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: null
	at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:126)
	at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:122)
	at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:373)
	at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1104)
	at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1054)
	at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:484)
	... 50 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error
	at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)
	at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)
	at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:494)
	at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:315)
	... 53 common frames omitted
Caused by: java.io.EOFException: null
	at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290)
	at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91)
	at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:164)
	at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:596)
	... 1 common frames omitted
 
2020-11-19 20:23:12 | INFO  | XNIO-1 task-7 | org.springframework.amqp.rabbit.connection.CachingConnectionFactory | Attempting to connect to: [192.168.17.193]
2020-11-19 20:23:12 | WARN  | AMQP Connection 192.168.17.193:5672 | com.rabbitmq.client.impl.ForgivingExceptionHandler | An unexpected connection driver error occured (Exception message: Connection reset)
2020-11-19 20:23:12 | WARN  | XNIO-1 task-7 | org.springframework.boot.actuate.amqp.RabbitHealthIndicator | Rabbit health check failed
org.springframework.amqp.AmqpIOException: java.io.IOException
	at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71)
	at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:530)
	at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:702)
	at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.createConnection(ConnectionFactoryUtils.java:214)
	at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2076)
	at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2050)
	at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2030)
	at org.springframework.boot.actuate.amqp.RabbitHealthIndicator.getVersion(RabbitHealthIndicator.java:49)
	at org.springframework.boot.actuate.amqp.RabbitHealthIndicator.doHealthCheck(RabbitHealthIndicator.java:44)
	at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:82)
	at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:95)
	at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:50)
	at org.springframework.boot.actuate.health.HealthEndpointWebExtension.health(HealthEndpointWebExtension.java:53)
	at sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282)
	at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:76)
	at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:60)
	at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$ServletWebOperationAdapter.handle(AbstractWebMvcEndpointHandlerMapping.java:278)
	at org.springframework.boot.actuate.endpoint.web.servlet.AbstractWebMvcEndpointHandlerMapping$OperationHandler.handle(AbstractWebMvcEndpointHandlerMapping.java:334)
	at sun.reflect.GeneratedMethodAccessor232.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1039)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:645)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
	at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
	at com.alibaba.druid.support.http.WebStatFilter.doFilter(WebStatFilter.java:124)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:88)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:92)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:118)
	at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
	at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
	at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
	at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
	at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
	at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
	at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
	at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132)
	at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
	at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
	at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
	at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
	at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269)
	at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130)
	at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
	at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249)
	at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78)
	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
	at org.apache.skywalking.apm.plugin.undertow.v2x.SWRunnable.run(SWRunnable.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: null
	at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:126)
	at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:122)
	at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:373)
	at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1104)
	at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1054)
	at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:484)
	... 95 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error
	at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)
	at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)
	at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:494)
	at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:315)
	... 98 common frames omitted
Caused by: java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
	at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:288)
	at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91)
	at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:164)
	at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:596)
	... 1 common frames omitted
2020-11-19 20:23:12 | INFO  | Thread-10 | com.alibaba.cloud.nacos.registry.NacosServiceRegistry | De-registering from Nacos Server now...
2020-11-19 20:23:12 | INFO  | Thread-10 | com.alibaba.cloud.nacos.registry.NacosServiceRegistry | De-registration finished.
2020-11-19 20:23:12 | INFO  | Thread-10 | org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor | Shutting down ExecutorService 'applicationTaskExecutor'
2020-11-19 20:23:12 | INFO  | Thread-10 | com.alibaba.druid.pool.DruidDataSource | {dataSource-1} closing ...
2020-11-19 20:23:12 | INFO  | Thread-10 | com.alibaba.druid.pool.DruidDataSource | {dataSource-1} closed
2020-11-19 20:23:12 | INFO  | Thread-10 | io.undertow.servlet | Destroying Spring FrameworkServlet 'dispatcherServlet'

查到报错：Rabbit health check failed， mq的健康检查未通过，导致容器/actuator/health 健康检查时显

{
    "status": "DOWN"
}

由于spring-boot-actuator 不太了解，查阅资料后得知，当项目中使用了mq 时，spring boot 自动生成mq的健康检查。mq无法连接时，导致整体的项目健康检查未通过，pod 重启

从前往后追从/actuator/health开始

package org.springframework.boot.actuate.health;
import ...

@Endpoint(id = "health")
public class HealthEndpoint {

	private final HealthIndicator healthIndicator;

	/**
	 * Create a new {@link HealthEndpoint} instance that will use the given
	 * {@code healthIndicator} to generate its response.
	 * @param healthIndicator the health indicator
	 */
	public HealthEndpoint(HealthIndicator healthIndicator) {
		Assert.notNull(healthIndicator, "HealthIndicator must not be null");
		this.healthIndicator = healthIndicator;
	}

	@ReadOperation
	public Health health() {
		return this.healthIndicator.health();
	}
    ...
}

由端点来提供接口并返回数据最终调用HealthIndicator .health()方法。

package org.springframework.boot.actuate.autoconfigure.health;

import ...


@Configuration
@ConditionalOnSingleCandidate(HealthIndicatorRegistry.class)
@ConditionalOnEnabledEndpoint(endpoint = HealthEndpoint.class)
class HealthEndpointConfiguration {

	@Bean
	@ConditionalOnMissingBean
	public HealthEndpoint healthEndpoint(HealthAggregator healthAggregator, HealthIndicatorRegistry registry) {
		return new HealthEndpoint(new CompositeHealthIndicator(healthAggregator, registry));
	}

}

由自动转装配的bean 对象实现了构造方法,查看一下CompositeHealthIndicator.health()实现

package org.springframework.boot.actuate.health;

import ...
public class CompositeHealthIndicator implements HealthIndicator {
...

	@Override
	public Health health() {
		Map<String, Health> healths = new LinkedHashMap<>();
		for (Map.Entry<String, HealthIndicator> entry : this.registry.getAll().entrySet()) {
			healths.put(entry.getKey(), entry.getValue().health());
		}
		return this.aggregator.aggregate(healths);
	}

}

遍历registry 中所有 对象获取health().HealthIndicatorRegistry 由下述方法实现

package org.springframework.boot.actuate.autoconfigure.health;

import ...
@Configuration
@EnableConfigurationProperties({ HealthIndicatorProperties.class })
public class HealthIndicatorAutoConfiguration {

...

	@Bean
	@ConditionalOnMissingBean(HealthIndicatorRegistry.class)
	public HealthIndicatorRegistry healthIndicatorRegistry(ApplicationContext applicationContext) {
		return HealthIndicatorRegistryBeans.get(applicationContext);
	}
...

}

get方法如下

package org.springframework.boot.actuate.autoconfigure.health;

import ...

final class HealthIndicatorRegistryBeans {

	private HealthIndicatorRegistryBeans() {
	}

	public static HealthIndicatorRegistry get(ApplicationContext applicationContext) {
		Map<String, HealthIndicator> indicators = new LinkedHashMap<>();
		indicators.putAll(applicationContext.getBeansOfType(HealthIndicator.class));
		if (ClassUtils.isPresent("reactor.core.publisher.Flux", null)) {
			new ReactiveHealthIndicators().get(applicationContext).forEach(indicators::putIfAbsent);
		}
		HealthIndicatorRegistryFactory factory = new HealthIndicatorRegistryFactory();
		return factory.createHealthIndicatorRegistry(indicators);
	}
...
}

从applicationContext中获取所有HealthIndicator对象。打断点后发现确实存在RabbitHealthIndicator 的bean 对象。

RabbitHealthIndicator.health()实现如下

package org.springframework.boot.actuate.amqp;

import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.boot.actuate.health.AbstractHealthIndicator;
import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;
import org.springframework.util.Assert;

/**
 * Simple implementation of a {@link HealthIndicator} returning status information for the
 * RabbitMQ messaging system.
 *
 * @author Christian Dupuis
 * @since 1.1.0
 */
public class RabbitHealthIndicator extends AbstractHealthIndicator {

	private final RabbitTemplate rabbitTemplate;

	public RabbitHealthIndicator(RabbitTemplate rabbitTemplate) {
		super("Rabbit health check failed");
		Assert.notNull(rabbitTemplate, "RabbitTemplate must not be null");
		this.rabbitTemplate = rabbitTemplate;
	}

	@Override
	protected void doHealthCheck(Health.Builder builder) throws Exception {
		builder.up().withDetail("version", getVersion());
	}

	private String getVersion() {
		return this.rabbitTemplate
				.execute((channel) -> channel.getConnection().getServerProperties().get("version").toString());
	}
    /****此方法由父类AbstractHealthIndicator 实现******/
    	@Override
	public final Health health() {
		Health.Builder builder = new Health.Builder();
		try {
			doHealthCheck(builder);
		}
		catch (Exception ex) {
			if (this.logger.isWarnEnabled()) {
				String message = this.healthCheckFailedMessage.apply(ex);
				this.logger.warn(StringUtils.hasText(message) ? message : DEFAULT_MESSAGE, ex);
			}
			builder.down(ex);
		}
		return builder.build();
	}    


}

可以看到当mq的健康检查未通过时，会有报错信息Rabbit health check failed，至此问题定位完毕。

4、解决问题

   如果mq 对项目影响较大，则由此健康检查保证项目对外服务时，是可用状态。

   如果mq的影响不大，健康检查可有可无，则可以通过配置来忽略mq的健康检查。

management.health.rabbit.enabled=false

此配置如何生效 可以查看参考文章《SpringBoot健康检查实现原理》，这里就不在赘述。

5、反思

    平时还需多查看源码，多积累，方能快速的定位问题。

 

 

参考文章：

Spring boot 2.0 Actuator 的健康检查

spring-boot-actuator中health的工作原理解析

SpringBoot健康检查实现原理