文章目录
本篇博文总结下, Consul 在SpringCloud框架下的高可用问题,及其他实践经验和踩坑记录。
转载请注明🙂, 喜欢请一键三连哦 😊
验证版本:
<java.version>1.8</java.version>
<spring-cloud.version>Hoxton.SR4</spring-cloud.version>
<spring-boot.version>2.2.6.RELEASE</spring-bootversion>
1. 服务实例问题
1.1 默认服务实例名称
- SpringCloud: 默认采用 服务名称+端口(不同SpringCloud版本可能不同), 但是SpringCloud推荐使用特殊的,比如使用SpringCloud instanceId(ip + port) + 随机数
1.2 自动注销问题
1.2.1 SpringCloud Consul 启用自动注销
SpringCloud: 可通过配置一下属性,使Consul 自动注销掉服务
deregister: false # 是否禁用自动注销服务功能, false 表示开启 自动注销功能(Disable automatic de-registration of service in consul.)
health-check-critical-timeout: 30s # 超过多长时间就注销非健康状态问题服务, 此处超过30秒,及注销掉服务。Timeout to deregister services critical for longer than timeout (e.g. 30m). Requires consul version 7.x or higher.
实现原理: https://www.consul.io/api/agent/check.html
注册服务时,支持 DeregisterCriticalServiceAfter 属性, SpringCloud增加配置后,就会在注册时,设置此属性, 比如 DeregisterCriticalServiceAfter:30s
1.3 Consul重复服务实例问题
TODO: 在其他博文中, 单独展开。
重复服务实例问题, 也就是同一个服务实例(同一Ip, port)在注册中心中出现了多次。 出现这种情况的原因,分为两种。
- 一种是每次注册时,使用了不同的实例ID(ServiceID), 比如随机数不同, 旧的服务实例没有下线,又注册了新的服务实例。
- 另外一种时,两次注册时,注册到不同的Server节点,即使同一实例名称(Server),也可以注册到不同的Server节点。
2. 原生SpringCloud 高可用问题
2.1 Consul集群挂掉后 : 基本无影响
表现:
- 使用本地缓存,请求正常
- 持续抛出连接 Consul 异常
2.2 Provider服务实例下线后: 基本无影响
表现:
- consumer短暂请求失败后, 请求恢复正常,请求全打到正常实例
全部下线后:
-
抛出无实例异常
-
Provider新增后,Consumer 调用恢复正常
2.3 Provider注册 Consul Client,Consul Client 挂掉: 不可用
-
数据直接丢失,数据不会同步到 Consul Server 节点
-
Consumer 直接扔出异常,[无可用实例]
2020-04-29 14:50:00.007 ERROR 19620 --- [ scheduling-1] o.s.s.s.TaskUtils$LoggingErrorHandler : Unexpected error occurred in scheduled task
java.lang.IllegalStateException: No instances available for spring-cloud-provider
at org.springframework.cloud.netflix.ribbon.RibbonLoadBalancerClient.execute(RibbonLoadBalancerClient.java:119) ~[spring-cloud-netflix-ribbon-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.cloud.netflix.ribbon.RibbonLoadBalancerClient.execute(RibbonLoadBalancerClient.java:99) ~[spring-cloud-netflix-ribbon-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.cloud.client.loadbalancer.LoadBalancerInterceptor.intercept(LoadBalancerInterceptor.java:58) ~[spring-cloud-commons-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:93) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.http.client.InterceptingClientHttpRequest.executeInternal(InterceptingClientHttpRequest.java:77) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:739) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:674) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:315) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at com.jdcloud.jdsf.demo.consumer.service.impl.RibbonProviderServiceImpl.autoRequest(RibbonProviderServiceImpl.java:96) ~[classes/:na]
at jdk.internal.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
- Provider 持续抛出连接异常
2020-04-29 14:53:07.986 ERROR 19104 --- [TaskScheduler-1] o.s.c.c.discovery.ConsulDiscoveryClient : Error watching Consul CatalogServices
com.ecwid.consul.transport.TransportException: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:8504 [/127.0.0.1] failed: Connection refused: connect
at com.ecwid.consul.transport.AbstractHttpTransport.executeRequest(AbstractHttpTransport.java:77) ~[consul-api-1.4.1.jar:na]
at com.ecwid.consul.transport.AbstractHttpTransport.makeGetRequest(AbstractHttpTransport.java:34) ~[consul-api-1.4.1.jar:na]
at com.ecwid.consul.v1.ConsulRawClient.makeGetRequest(ConsulRawClient.java:128) ~[consul-api-1.4.1.jar:na]
at com.ecwid.consul.v1.catalog.CatalogConsulClient.getCatalogServices(CatalogConsulClient.java:120) ~[consul-api-1.4.1.jar:na]
at com.ecwid.consul.v1.ConsulClient.getCatalogServices(ConsulClient.java:372) ~[consul-api-1.4.1.jar:na]
at org.springframework.cloud.consul.discovery.ConsulCatalogWatch.catalogServicesWatch(ConsulCatalogWatch.java:129) ~[spring-cloud-consul-discovery-2.2.2.RELEASE.jar:2.2.2.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[na:na]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:8504 [/127.0.0.1] failed: Connection refused: connect
2.4 健康检查失败后,被摘掉实例 :无法重新注册
- 缺少重新注册逻辑,或者我们可以使用默认的机制,停止应用后,不对服务实例进行剔除;
2.5 Consul Node重启后,若数据丢失 :无法重新注册
SpringCloud Consul 默认仅在启动时发起注册, 我们应该保证Consul 节点重启后, 数据不会丢失;
3. 其他常见问题
3.1 拉取服务实例问题
若: 注册时,不设置Chcek, 拉去Passing服务时, 也会被拉去到