dubbo服务注册成功，消费者报错No provider available 原因探索

报错：

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
[2020-01-07 23:39:52.098] [main] [ERROR] [org.springframework.boot.SpringApplication.reportFailure:822] - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'paymentServiceImpl' defined in URL [*******************/PaymentServiceImpl.class]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: paymentServiceImpl; nested exception is java.lang.IllegalStateException: Failed to check the status of the *********************.SegChangeService. No provider available for the service **************.SegChangeService from the url registry://***.***.***.***:2181/com.alibaba.dubbo.registry.RegistryService?application=consumer&cluster=available&dubbo=2.6.0&pid=39858&refer=application%3Dconsumer%26dubbo%3D2.6.0%26interface%3D**********.SegChangeService%26methods%3DverifyBlackList%2CprocessChangeBackGroundSplit%2CcreateChange%2CprocessIntlSegChange%2CstopChange%2CprocessChangeBackGround%2CprocessVolChangeBackGround%2Ce2ESearchChangeInfo%2CprocessIndFlightChange%2CsearchChangeVariant%2ClogETSError%2CsearchChange%2CsearchChangeAllChannel%2CdirectChangeSeg%2CgetFlightInfoByChangeNo%2CsendMsg%2CprocessVolChange%2CapplyDomesticSegChange%2CafterSalePnrCheck%2CgetMarketingFlightNo%2CretrieveLog%2CsendMessage%2CprocessDomesticSegChange%2CretrieveChangeItemEMP%2CpnrCheck%2CprocessVolChangeBackGroundSplit%2CcheckRtCode%2CprocessChange%2CfindOtherChangeByTicketNo%2CsearchChangeAllChannelInPage%2CgetChangeOrderInfo%2CretrieveChangeItem%2CquickSearch%2CchangeChangeStatus%2CsimpleSearch%2CapplyIntlSegChange%2CchangeRetry%2CcreateChangePayBill%26pid%3D39858%26register.ip%3D***.***.***.***%26side%3Dconsumer%26timestamp%3D1578411585978&registry=zookeeper&timestamp=1578411586009 to the consumer ***.***.***.*** use dubbo version 2.6.0
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:601)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:845)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
        at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:743)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:390)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1214)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1203)
        at *********Application.main(****Application.java:13)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: org.springframework.beans.factory.BeanCreationException: paymentServiceImpl; nested exception is java.lang.IllegalStateException: Failed to check the status of the service ************.SegChangeService. No provider available for the service **************.SegChangeService from the url registry://***.***.***.***:2181/com.alibaba.dubbo.registry.RegistryService?application=consumer&cluster=available&dubbo=2.6.0&pid=39858&refer=application%3Dconsumer%26dubbo%3D2.6.0%26interface%3D*************.SegChangeService%26methods%3DverifyBlackList%2CprocessChangeBackGroundSplit%2CcreateChange%2CprocessIntlSegChange%2CstopChange%2CprocessChangeBackGround%2CprocessVolChangeBackGround%2Ce2ESearchChangeInfo%2CprocessIndFlightChange%2CsearchChangeVariant%2ClogETSError%2CsearchChange%2CsearchChangeAllChannel%2CdirectChangeSeg%2CgetFlightInfoByChangeNo%2CsendMsg%2CprocessVolChange%2CapplyDomesticSegChange%2CafterSalePnrCheck%2CgetMarketingFlightNo%2CretrieveLog%2CsendMessage%2CprocessDomesticSegChange%2CretrieveChangeItemEMP%2CpnrCheck%2CprocessVolChangeBackGroundSplit%2CcheckRtCode%2CprocessChange%2CfindOtherChangeByTicketNo%2CsearchChangeAllChannelInPage%2CgetChangeOrderInfo%2CretrieveChangeItem%2CquickSearch%2CchangeChangeStatus%2CsimpleSearch%2CapplyIntlSegChange%2CchangeRetry%2CcreateChangePayBill%26pid%3D39858%26register.ip%3D***.***.***.***%26side%3Dconsumer%26timestamp%3D1578411585978&registry=zookeeper&timestamp=1578411586009 to the consumer ***.***.***.*** use dubbo version 2.6.0
        at com.alibaba.dubbo.spring.boot.DubboConsumerAutoConfiguration$1.postProcessBeforeInitialization(DubboConsumerAutoConfiguration.java:100)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1770)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
        ... 23 common frames omitted
Caused by: java.lang.IllegalStateException: Failed to check the status of the service ***************.SegChangeService. No provider available for the service ***************.SegChangeService from the url registry://***.***.***.***:2181/com.alibaba.dubbo.registry.RegistryService?application=consumer&cluster=available&dubbo=2.6.0&pid=39858&refer=application%3Dconsumer%26dubbo%3D2.6.0%26interface%3D*******************.SegChangeService%26methods%3DverifyBlackList%2CprocessChangeBackGroundSplit%2CcreateChange%2CprocessIntlSegChange%2CstopChange%2CprocessChangeBackGround%2CprocessVolChangeBackGround%2Ce2ESearchChangeInfo%2CprocessIndFlightChange%2CsearchChangeVariant%2ClogETSError%2CsearchChange%2CsearchChangeAllChannel%2CdirectChangeSeg%2CgetFlightInfoByChangeNo%2CsendMsg%2CprocessVolChange%2CapplyDomesticSegChange%2CafterSalePnrCheck%2CgetMarketingFlightNo%2CretrieveLog%2CsendMessage%2CprocessDomesticSegChange%2CretrieveChangeItemEMP%2CpnrCheck%2CprocessVolChangeBackGroundSplit%2CcheckRtCode%2CprocessChange%2CfindOtherChangeByTicketNo%2CsearchChangeAllChannelInPage%2CgetChangeOrderInfo%2CretrieveChangeItem%2CquickSearch%2CchangeChangeStatus%2CsimpleSearch%2CapplyIntlSegChange%2CchangeRetry%2CcreateChangePayBill%26pid%3D39858%26register.ip%3D***.***.***.***%26side%3Dconsumer%26timestamp%3D1578411585978&registry=zookeeper&timestamp=1578411586009 to the consumer ***.***.***.*** use dubbo version 2.6.0
        at com.alibaba.dubbo.config.ReferenceConfig.createProxy(ReferenceConfig.java:422)
        at com.alibaba.dubbo.config.ReferenceConfig.init(ReferenceConfig.java:333)
        at com.alibaba.dubbo.config.ReferenceConfig.get(ReferenceConfig.java:163)
        at com.alibaba.dubbo.config.spring.ReferenceBean.getObject(ReferenceBean.java:59)
        at com.alibaba.dubbo.spring.boot.DubboConsumerAutoConfiguration$1.postProcessBeforeInitialization(DubboConsumerAutoConfiguration.java:89)
        ... 26 common frames omitted

 

场景描述：

接口服务器通过zookeeper+dubbo调后台服务，原先有两台服务器A和B提供后台服务，其中B服务器部署有zookeeper，两个后台服务都注册到B上的zookeeper，基本架构如图所示：

后来因一些原因后台切换了IP地址，并且架构也一点改变。改变后架构图如下：

接口切换到新的后台地址发布，就出问题了，抛出了上面的报错信息。

 

原因分析：

1. 新的后台缺少某些接口调用的服务。telnet新后台服务器ip的20880端口，可以发现报错信息中的SegChangeService这个服务有成功注册到zookeeper。排除。

2. 原来接口在用的某些方法在新的后台服务中缺失。排查了后台的war包，发现接口依赖的方法都是有的。排除。

3. 接口依赖的jar包还是旧后台提供的，是否与jar包有关系。替换了新后台的jar包，发布时报同样的错误。

4. 考虑是否与网络关系有关。接口只开通了zookeeper服务器的2181端口和20880端口，没有开通到后台服务器的网络关系。(o゜▽゜)o☆[BINGO!] 答对了。正确做法是开通接口服务器到zookeeper服务器的2181端口和开通接口服务器到后台服务器的20880端口。

 

总结：

这个问题折腾了几天。总的来说是对dubbo和zookeeper理解不到位，没有理解清楚后台服务是通过20880把服务暴露给zookeeper，zookeeper通过2181端口对消费者暴露服务地址，zookeeper只是一个转发的作用，最终消费者还是会跟后台服务器建立起连接。