Cassandra 集群使用Gossip协议维护集群状态 各个节点之间是平等的 并非如同hadoop一样存在主从之分
通过Gossip协议可以知道集群中各个节点的当前状态
Cassandra中的GossipTask类主要实现了节点间消息通信机制
// wait on messaging service to start listening
MessagingService.instance().waitUntilListening();
/* Update the local heartbeat counter. */
endpointStateMap.get(FBUtilities.getBroadcastAddress())
.getHeartBeatState().updateHeartBeat();
final List<GossipDigest> gDigests = new ArrayList<GossipDigest>();
Gossiper.instance.makeRandomGossipDigest(gDigests);
if (gDigests.size() > 0)
{
GossipDigestSyn digestSynMessage = new GossipDigestSyn(
DatabaseDescriptor.getClusterName(),
DatabaseDescriptor.getPartitionerName(),
gDigests);
MessageOut<GossipDigestSyn> message = new MessageOut<GossipDigestSyn>(
MessagingService.Verb.GOSSIP_DIGEST_SYN,
digestSynMessage,
GossipDigestSyn.serializer);
/* Gossip to some random live member */
boolean gossipedToSeed = doGossipToLiveMember(message);
/* Gossip to some unreachable member to check if he is back up */
doGossipToUnreachableMember(message);
if (!gossipedToSeed || liveEndpoints.size() < seeds.size())
doGossipToSeed(message);
doStatusCheck();
}
看看sendGossip方法的是实现:
List<InetAddress> liveEndpoints = ImmutableList.copyOf(epSet);
int size = liveEndpoints.size();
...
/* Generate a random number from 0 -> size */
int index = (size == 1) ? 0 : random.nextInt(size);
InetAddress to = liveEndpoints.get(index);
MessagingService.instance().sendOneWay(message, to);
MessagingService sendOneWay方法实现部分
主要部分如下所示 简单说就是先获取一个连接再把需要发送的连接放进队列
其中connectionManagers是一个 ConcurrentMap<InetAddress, OutboundTcpConnectionPool>
用来存储Address到ConnectionPool的映射表
enqueue的时候会先检查队列长度 大于1024 会触发expireMessages机制
另队列中的消息包装成了QueuedMessage结构
// get pooled connection (really, connection queue)
OutboundTcpConnection connection = getConnection(to, processedMessage);
// write it
connection.enqueue(processedMessage, id);
SinkManager类主要用来做message集合使用
Gossip 协议交换过程
节点在开始交换Gossip信息的时候 首先发送一个GossipDigestSynMessage
GossipDigestSyn digestSynMessage = new GossipDigestSyn(
DatabaseDescriptor.getClusterName(),
DatabaseDescriptor.getPartitionerName(),gDigests);
MessageOut<GossipDigestSyn> message = new MessageOut<GossipDigestSyn>(
MessagingService.Verb.GOSSIP_DIGEST_SYN,
digestSynMessage,GossipDigestSyn.serializer);
当有节点收到GossipDigestSynMessage时 使用GossipDigestAckMessage回复消息
GossipDigestAckMessage其中主要包含两部分:GossipDigest列表和InetAddress --> EndpointState映射
final List<GossipDigest> gDigestList;
final Map<InetAddress, EndpointState> epStateMap;
具体的处理过程以IVerbHandler的形式注册到MessagingService上
verbHandler.doVerb(message, id);// 具体调用相应MessageHandler
参考资料:
[ArchitectureGossip] http://wiki.apache.org/cassandra/ArchitectureGossip
[internode communication] http://www.datastax.com/documentation/cassandra/1.2/cassandra/architecture/architectureGossipAbout_c.html
[Cassandra中Gossip具体实现方式] http://blog.csdn.net/zhangzhaokun/article/details/5859760