Kafka连接问题，导致Spark数据分析中断

Spark Streaming流式计算经常会中断，中断时Kafka日志如下：

[2016-10-08 15:14:02,518] INFO re-registering broker info in ZK for broker 0 (kafka.server.KafkaHealthcheck)

[2016-10-08 15:14:02,531] INFO Registered broker 0 at path /brokers/ids/0 with address 30.209.81.2:9092. (kafka.utils.ZkUtils$)
[2016-10-08 15:14:02,531] INFO done re-registering broker (kafka.server.KafkaHealthcheck)
[2016-10-08 15:14:02,532] INFO Subscribing to /brokers/topics path to watch for new topics (kafka.server.KafkaHealthcheck)
[2016-10-08 15:14:02,601] ERROR Closing socket for /30.209.81.2 because of error (kafka.network.Processor)
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcher.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
        at sun.nio.ch.IOUtil.read(IOUtil.java:171)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245)
        at kafka.utils.Utils$.read(Utils.scala:380)
        at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
        at kafka.network.Processor.read(SocketServer.scala:444)
        at kafka.network.Processor.run(SocketServer.scala:340)
        at java.lang.Thread.run(Thread.java:662)
[2016-10-08 15:14:02,738] INFO Closing socket connection to /30.209.81.2. (kafka.network.Processor)
[2016-10-08 15:14:02,800] INFO Closing socket connection to /30.209.81.2. (kafka.network.Processor)
[2016-10-08 15:14:02,908] ERROR Closing socket for /30.209.81.2 because of error (kafka.network.Processor)
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcher.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
        at sun.nio.ch.IOUtil.read(IOUtil.java:171)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245)
        at kafka.utils.Utils$.read(Utils.scala:380)
        at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
        at kafka.network.Processor.read(SocketServer.scala:444)
        at kafka.network.Processor.run(SocketServer.scala:340)

        at java.lang.Thread.run(Thread.java:662)

此时三台ZK的日志如下：

2016-10-08 15:12:33,948 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1001] - Closed socket connection for client /30.209.81.2:55750 which had sessionid 0x255de68ef894c0d
2016-10-08 15:12:33,948 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /30.209.81.36:2577
2016-10-08 15:12:33,949 [myid:2] - INFO  [Thread-2154098:NIOServerCnxn@1001] - Closed socket connection for client /30.209.81.11:13811 (no session established for client)
2016-10-08 15:12:33,949 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@821] - Processing stat command from /30.209.81.11:13824
2016-10-08 15:12:52,604 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /30.209.81.11:13836
2016-10-08 15:12:52,604 [myid:2] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@349] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x55de68ef7f4bcd, likely client has closed socket
        at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
        at java.lang.Thread.run(Thread.java:662)
2016-10-08 15:12:52,605 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1001] - Closed socket connection for client /30.209.81.11:13784 which had sessionid 0x55de68ef7f4bcd
2016-10-08 15:12:52,605 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@839] - Client attempting to establish new session at /30.209.81.36:2577
2016-10-08 15:12:52,605 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /30.209.81.37:33647
2016-10-08 15:12:52,605 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@821] - Processing stat command from /30.209.81.11:13836
2016-10-08 15:12:52,606 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /30.209.81.11:13854
2016-10-08 15:12:52,606 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@832] - Client attempting to renew session 0x155de68ef934fd9 at /30.209.81.37:33647
2016-10-08 15:12:52,606 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:Learner@107] - Revalidating client: 0x155de68ef934fd9
2016-10-08 15:12:52,607 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /30.209.81.11:13868
2016-10-08 15:12:52,607 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@821] - Processing stat command from /30.209.81.11:13854
2016-10-08 15:12:52,607 [myid:2] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@588] - Invalid session 0x155de68ef934fd9 for client /30.209.81.37:33647, probably expired
2016-10-08 15:12:52,607 [myid:2] - INFO  [Thread-2154100:NIOServerCnxn$StatCommand@655] - Stat command output
2016-10-08 15:12:52,608 [myid:2] - INFO  [Thread-2154100:NIOServerCnxn@1001] - Closed socket connection for client /30.209.81.11:13836 (no session established for client)
2016-10-08 15:12:52,610 [myid:2] - INFO  [Thread-2154099:NIOServerCnxn$StatCommand@655] - Stat command output
2016-10-08 15:12:52,610 [myid:2] - ERROR [CommitProcessor:2:NIOServerCnxn@180] - Unexpected Exception: 
java.nio.channels.CancelledKeyException
        at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
        at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
        at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:153)
        at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1076)
        at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
        at org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:74)
2016-10-08 15:12:52,618 [myid:2] - INFO  [Thread-2154101:NIOServerCnxn$StatCommand@655] - Stat command output
2016-10-08 15:12:52,618 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /30.209.81.11:13881
2016-10-08 15:12:52,619 [myid:2] - INFO  [Thread-2154099:NIOServerCnxn@1001] - Closed socket connection for client /30.209.81.11:13824 (no session established for client)
2016-10-08 15:12:52,620 [myid:2] - INFO  [CommitProcessor:2:ZooKeeperServer@595] - Established session 0x255de68ef895d5e with negotiated timeout 18000 for client /30.209.81.36:2577

网上有说三台ZK配置不一致导致的：http://community.cloudera.com/t5/CDH-Manual-Installation/FATAL-ha-ZKFailoverController-Unable-to-start-failover/td-p/16000

并通过修改配置解决，但检查本地集群后发现，配置项目，并不存在该问题，遗留问题，待解决。

如果有朋友遇到相同问题且知道解决方法，请留言，谢谢！