【Docker容器启动问题】容器启动时， exceptions.JedisConnectionException:Could not get a resource from the pool

问题现场环境：

1、本地虚拟机CentOS7 下的docker环境。

2、docker下的redis、mysql已正常启动，且虚拟机外可正常访问。

3、启动容器(SpringCloud 网关服务)需要使用 docker下的redis、MySql。

 

---

问题现象：

在启动容器时，出现错误异常c.n.d.s.t.d.RedirectingEurekaHttpClient：Request execution error和nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool，如下所示：

2018-09-13 12:53:25.742 ERROR 1 --- [           main] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error

com.sun.jersey.api.client.ClientHandlerException: java.net.NoRouteToHostException: No route to host (Host unreachable)
	at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187) ~[jersey-apache-client4-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27) ~[eureka-client-1.4.12.jar!/:1.4.12]
	at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) ~[jersey-client-1.19.1.jar!/:1.19.1]
	at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.getApplicationsInternal(AbstractJerseyEurekaHttpClient.java:194) ~[eureka-client-1.4.12.jar!/:1.4.12]

……

Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
2018-09-13 12:53:29.150 ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application startup failed

org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.cloud.netflix.eureka.EurekaDiscoveryClientConfiguration'; nested exception is org.springframework.data
.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:50) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:348) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:151) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:114) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:879) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.finishRefresh(EmbeddedWebApplicationContext.java:144) ~[spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:545) ~[spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) ~[spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1162) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1151) [spring-boot-1.5.2.RELEASE.jar!/:1.5.2.RELEASE]
	at com.wondersoft.gateway.GatewayApplication.main(GatewayApplication.java:31) [classes!/:1.5.2.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_111]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_111]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_111]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_111]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]
	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [gateway-1.5.2.RELEASE.jar:1.5.2.RELEASE]

 

---

问题分析：

1、异常c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error

com.sun.jersey.api.client.ClientHandlerException: java.net.NoRouteToHostException: No route to host (Host unreachable)，是处于网关在连接另外一个服务(注册中心)时，抛出的异常，属于无法正常路由连接到注册中心服务。

2、异常nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool，是处于正在连接redis时，抛出的异常，属于无法连接redis。

基于上述两点，问题属于容器间网络通信问题。

 

---

解决方法：

1、关闭防火墙。

使用root用户，执行  systemctl stop firewalld.service 命令，关闭防火墙。

 

2、重新启动 网关服务容器，正常启动，则问题解决。

3、如果启动时，出现错误

iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8111 -j DNAT，如下图：

 

[root@cen7 data]# docker start gateway
Error response from daemon: Cannot start container gateway: iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 8111 -j DNAT --to-destination 172.17.0.5:8111 ! -i docker0: iptables: No chain/target/match by that name.
 (exit status 1)
Error: failed to start containers: [gateway]

则需要重建docker0网络恢复，按顺序执行如下命令，即可。

pkill docker 
iptables -t nat -F 
ifconfig docker0 down 
brctl delbr docker0 
docker -d 
service docker restart

执行过程如下：

[root@cen7 data]# pkill docker 
[root@cen7 data]# iptables -t nat -F 
[root@cen7 data]# ifconfig docker0 down 
[root@cen7 data]# brctl delbr docker0 
[root@cen7 data]# docker -d 
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock) 
INFO[0000] [graphdriver] using prior storage driver "devicemapper" 
WARN[0000] Running modprobe bridge nf_nat failed with message: , error: exit status 1 
INFO[0000] Loading containers: start.                   
..............
INFO[0000] Loading containers: done.                    
INFO[0000] Daemon has completed initialization          
INFO[0000] Docker daemon                                 commit=786b29d execdriver=native-0.2 graphdriver=devicemapper version=1.7.1
^CINFO[0061] Processing signal 'interrupt'                
[root@cen7 data]# systemctl restart docker

4、启动所需容器即可。