Consul高可用场景分析及实践总结—SpringCloud篇

Consul-LOGO

本篇博文总结下, Consul 在SpringCloud框架下的高可用问题,及其他实践经验和踩坑记录
转载请注明🙂, 喜欢请一键三连哦 😊

验证版本:

    <java.version>1.8</java.version>
    <spring-cloud.version>Hoxton.SR4</spring-cloud.version>
    <spring-boot.version>2.2.6.RELEASE</spring-bootversion>

1. 服务实例问题

1.1 默认服务实例名称

  • SpringCloud: 默认采用 服务名称+端口(不同SpringCloud版本可能不同), 但是SpringCloud推荐使用特殊的,比如使用SpringCloud instanceId(ip + port) + 随机数

1.2 自动注销问题

1.2.1 SpringCloud Consul 启用自动注销

SpringCloud: 可通过配置一下属性,使Consul 自动注销掉服务


deregister: false  #  是否禁用自动注销服务功能, false 表示开启 自动注销功能(Disable automatic de-registration of service in consul.)
health-check-critical-timeout: 30s   # 超过多长时间就注销非健康状态问题服务, 此处超过30秒,及注销掉服务。Timeout to deregister services critical for longer than timeout (e.g. 30m). Requires consul version 7.x or higher.

实现原理: https://www.consul.io/api/agent/check.html

注册服务时,支持 DeregisterCriticalServiceAfter 属性, SpringCloud增加配置后,就会在注册时,设置此属性, 比如 DeregisterCriticalServiceAfter:30s

1.3 Consul重复服务实例问题

TODO: 在其他博文中, 单独展开。

重复服务实例问题, 也就是同一个服务实例(同一Ip, port)在注册中心中出现了多次。 出现这种情况的原因,分为两种。

  • 一种是每次注册时,使用了不同的实例ID(ServiceID), 比如随机数不同, 旧的服务实例没有下线,又注册了新的服务实例。
  • 另外一种时,两次注册时,注册到不同的Server节点,即使同一实例名称(Server),也可以注册到不同的Server节点。

2. 原生SpringCloud 高可用问题

2.1 Consul集群挂掉后 : 基本无影响

表现:

  • 使用本地缓存,请求正常
  • 持续抛出连接 Consul 异常

2.2 Provider服务实例下线后: 基本无影响

表现:

  • consumer短暂请求失败后, 请求恢复正常,请求全打到正常实例

全部下线后:

  • 抛出无实例异常

  • Provider新增后,Consumer 调用恢复正常

2.3 Provider注册 Consul Client,Consul Client 挂掉: 不可用

  • 数据直接丢失,数据不会同步到 Consul Server 节点

  • Consumer 直接扔出异常,[无可用实例]

2020-04-29 14:50:00.007 ERROR 19620 --- [   scheduling-1] o.s.s.s.TaskUtils$LoggingErrorHandler    : Unexpected error occurred in scheduled task

java.lang.IllegalStateException: No instances available for spring-cloud-provider
	at org.springframework.cloud.netflix.ribbon.RibbonLoadBalancerClient.execute(RibbonLoadBalancerClient.java:119) ~[spring-cloud-netflix-ribbon-2.2.2.RELEASE.jar:2.2.2.RELEASE]
	at org.springframework.cloud.netflix.ribbon.RibbonLoadBalancerClient.execute(RibbonLoadBalancerClient.java:99) ~[spring-cloud-netflix-ribbon-2.2.2.RELEASE.jar:2.2.2.RELEASE]
	at org.springframework.cloud.client.loadbalancer.LoadBalancerInterceptor.intercept(LoadBalancerInterceptor.java:58) ~[spring-cloud-commons-2.2.2.RELEASE.jar:2.2.2.RELEASE]
	at org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:93) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at org.springframework.http.client.InterceptingClientHttpRequest.executeInternal(InterceptingClientHttpRequest.java:77) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:739) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:674) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:315) ~[spring-web-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at com.jdcloud.jdsf.demo.consumer.service.impl.RibbonProviderServiceImpl.autoRequest(RibbonProviderServiceImpl.java:96) ~[classes/:na]
	at jdk.internal.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]


  • Provider 持续抛出连接异常
2020-04-29 14:53:07.986 ERROR 19104 --- [TaskScheduler-1] o.s.c.c.discovery.ConsulDiscoveryClient  : Error watching Consul CatalogServices

com.ecwid.consul.transport.TransportException: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:8504 [/127.0.0.1] failed: Connection refused: connect
	at com.ecwid.consul.transport.AbstractHttpTransport.executeRequest(AbstractHttpTransport.java:77) ~[consul-api-1.4.1.jar:na]
	at com.ecwid.consul.transport.AbstractHttpTransport.makeGetRequest(AbstractHttpTransport.java:34) ~[consul-api-1.4.1.jar:na]
	at com.ecwid.consul.v1.ConsulRawClient.makeGetRequest(ConsulRawClient.java:128) ~[consul-api-1.4.1.jar:na]
	at com.ecwid.consul.v1.catalog.CatalogConsulClient.getCatalogServices(CatalogConsulClient.java:120) ~[consul-api-1.4.1.jar:na]
	at com.ecwid.consul.v1.ConsulClient.getCatalogServices(ConsulClient.java:372) ~[consul-api-1.4.1.jar:na]
	at org.springframework.cloud.consul.discovery.ConsulCatalogWatch.catalogServicesWatch(ConsulCatalogWatch.java:129) ~[spring-cloud-consul-discovery-2.2.2.RELEASE.jar:2.2.2.RELEASE]
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.2.5.RELEASE.jar:5.2.5.RELEASE]
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[na:na]
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:8504 [/127.0.0.1] failed: Connection refused: connect

2.4 健康检查失败后,被摘掉实例 :无法重新注册

  1. 缺少重新注册逻辑,或者我们可以使用默认的机制,停止应用后,不对服务实例进行剔除;

2.5 Consul Node重启后,若数据丢失 :无法重新注册

SpringCloud Consul 默认仅在启动时发起注册, 我们应该保证Consul 节点重启后, 数据不会丢失;

3. 其他常见问题

3.1 拉取服务实例问题

若: 注册时,不设置Chcek, 拉去Passing服务时, 也会被拉去到

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值