准备阶段:提议者向接受者发送一个提议编号,接受者如果收到的提议编号比自己已经接受的提议编号大,接受该提议,并返回自己已经接受的提议编号和值
提议阶段:提议者收到了大多数接受者的接受,提议者向所有接受者发送一个提议,包括提议编号和值。
学习阶段:一旦某值被大多数接受者接受,学习者就可以学习到这个值并应用到系统中
#mermaid-svg-wpPHN1YFrSOobcFY {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-wpPHN1YFrSOobcFY .error-icon{fill:#552222;}#mermaid-svg-wpPHN1YFrSOobcFY .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-wpPHN1YFrSOobcFY .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-wpPHN1YFrSOobcFY .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-wpPHN1YFrSOobcFY .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-wpPHN1YFrSOobcFY .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-wpPHN1YFrSOobcFY .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-wpPHN1YFrSOobcFY .marker{fill:#333333;stroke:#333333;}#mermaid-svg-wpPHN1YFrSOobcFY .marker.cross{stroke:#333333;}#mermaid-svg-wpPHN1YFrSOobcFY svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-wpPHN1YFrSOobcFY .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-wpPHN1YFrSOobcFY .cluster-label text{fill:#333;}#mermaid-svg-wpPHN1YFrSOobcFY .cluster-label span{color:#333;}#mermaid-svg-wpPHN1YFrSOobcFY .label text,#mermaid-svg-wpPHN1YFrSOobcFY span{fill:#333;color:#333;}#mermaid-svg-wpPHN1YFrSOobcFY .node rect,#mermaid-svg-wpPHN1YFrSOobcFY .node circle,#mermaid-svg-wpPHN1YFrSOobcFY .node ellipse,#mermaid-svg-wpPHN1YFrSOobcFY .node polygon,#mermaid-svg-wpPHN1YFrSOobcFY .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-wpPHN1YFrSOobcFY .node .label{text-align:center;}#mermaid-svg-wpPHN1YFrSOobcFY .node.clickable{cursor:pointer;}#mermaid-svg-wpPHN1YFrSOobcFY .arrowheadPath{fill:#333333;}#mermaid-svg-wpPHN1YFrSOobcFY .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-wpPHN1YFrSOobcFY .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-wpPHN1YFrSOobcFY .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-wpPHN1YFrSOobcFY .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-wpPHN1YFrSOobcFY .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-wpPHN1YFrSOobcFY .cluster text{fill:#333;}#mermaid-svg-wpPHN1YFrSOobcFY .cluster span{color:#333;}#mermaid-svg-wpPHN1YFrSOobcFY div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-wpPHN1YFrSOobcFY :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
接受者收到提案者空白提案请求时的原则
不在接受提案ID小于等于当前收到的
不在接受提案落档中提案ID小于当前的
不违背以前做出的承诺
Paxos算法完美情况
Paxos算法弊端
ZAB协议算法 - Paxos算法的改良 - 集群仅能一位提议者(即Leader)
认识
概念: 只有一台客户端(Leader)负责处理外部的写事务请求,然后Leader客户端将数据同步到其他Follower节点。即Zookeeper只有一个Leader可以发起提案
#mermaid-svg-lBpFvnCpVyIVlq0b {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-lBpFvnCpVyIVlq0b .error-icon{fill:#552222;}#mermaid-svg-lBpFvnCpVyIVlq0b .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-lBpFvnCpVyIVlq0b .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-lBpFvnCpVyIVlq0b .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-lBpFvnCpVyIVlq0b .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-lBpFvnCpVyIVlq0b .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-lBpFvnCpVyIVlq0b .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-lBpFvnCpVyIVlq0b .marker{fill:#333333;stroke:#333333;}#mermaid-svg-lBpFvnCpVyIVlq0b .marker.cross{stroke:#333333;}#mermaid-svg-lBpFvnCpVyIVlq0b svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-lBpFvnCpVyIVlq0b .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-lBpFvnCpVyIVlq0b .cluster-label text{fill:#333;}#mermaid-svg-lBpFvnCpVyIVlq0b .cluster-label span{color:#333;}#mermaid-svg-lBpFvnCpVyIVlq0b .label text,#mermaid-svg-lBpFvnCpVyIVlq0b span{fill:#333;color:#333;}#mermaid-svg-lBpFvnCpVyIVlq0b .node rect,#mermaid-svg-lBpFvnCpVyIVlq0b .node circle,#mermaid-svg-lBpFvnCpVyIVlq0b .node ellipse,#mermaid-svg-lBpFvnCpVyIVlq0b .node polygon,#mermaid-svg-lBpFvnCpVyIVlq0b .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-lBpFvnCpVyIVlq0b .node .label{text-align:center;}#mermaid-svg-lBpFvnCpVyIVlq0b .node.clickable{cursor:pointer;}#mermaid-svg-lBpFvnCpVyIVlq0b .arrowheadPath{fill:#333333;}#mermaid-svg-lBpFvnCpVyIVlq0b .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-lBpFvnCpVyIVlq0b .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-lBpFvnCpVyIVlq0b .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-lBpFvnCpVyIVlq0b .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-lBpFvnCpVyIVlq0b .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-lBpFvnCpVyIVlq0b .cluster text{fill:#333;}#mermaid-svg-lBpFvnCpVyIVlq0b .cluster span{color:#333;}#mermaid-svg-lBpFvnCpVyIVlq0b div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-lBpFvnCpVyIVlq0b :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
ZAB模式
消息广播
崩溃恢复
#mermaid-svg-lCn4CniYSaXo93Ee {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-lCn4CniYSaXo93Ee .error-icon{fill:#552222;}#mermaid-svg-lCn4CniYSaXo93Ee .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-lCn4CniYSaXo93Ee .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-lCn4CniYSaXo93Ee .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-lCn4CniYSaXo93Ee .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-lCn4CniYSaXo93Ee .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-lCn4CniYSaXo93Ee .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-lCn4CniYSaXo93Ee .marker{fill:#333333;stroke:#333333;}#mermaid-svg-lCn4CniYSaXo93Ee .marker.cross{stroke:#333333;}#mermaid-svg-lCn4CniYSaXo93Ee svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-lCn4CniYSaXo93Ee .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-lCn4CniYSaXo93Ee .cluster-label text{fill:#333;}#mermaid-svg-lCn4CniYSaXo93Ee .cluster-label span{color:#333;}#mermaid-svg-lCn4CniYSaXo93Ee .label text,#mermaid-svg-lCn4CniYSaXo93Ee span{fill:#333;color:#333;}#mermaid-svg-lCn4CniYSaXo93Ee .node rect,#mermaid-svg-lCn4CniYSaXo93Ee .node circle,#mermaid-svg-lCn4CniYSaXo93Ee .node ellipse,#mermaid-svg-lCn4CniYSaXo93Ee .node polygon,#mermaid-svg-lCn4CniYSaXo93Ee .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-lCn4CniYSaXo93Ee .node .label{text-align:center;}#mermaid-svg-lCn4CniYSaXo93Ee .node.clickable{cursor:pointer;}#mermaid-svg-lCn4CniYSaXo93Ee .arrowheadPath{fill:#333333;}#mermaid-svg-lCn4CniYSaXo93Ee .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-lCn4CniYSaXo93Ee .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-lCn4CniYSaXo93Ee .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-lCn4CniYSaXo93Ee .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-lCn4CniYSaXo93Ee .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-lCn4CniYSaXo93Ee .cluster text{fill:#333;}#mermaid-svg-lCn4CniYSaXo93Ee .cluster span{color:#333;}#mermaid-svg-lCn4CniYSaXo93Ee div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-lCn4CniYSaXo93Ee :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
ZAB针对事务处理过程(两阶段)
广播事务阶段
广播事务提交操作
#mermaid-svg-lzCFF8NlsyxNreYy {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-lzCFF8NlsyxNreYy .error-icon{fill:#552222;}#mermaid-svg-lzCFF8NlsyxNreYy .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-lzCFF8NlsyxNreYy .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-lzCFF8NlsyxNreYy .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-lzCFF8NlsyxNreYy .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-lzCFF8NlsyxNreYy .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-lzCFF8NlsyxNreYy .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-lzCFF8NlsyxNreYy .marker{fill:#333333;stroke:#333333;}#mermaid-svg-lzCFF8NlsyxNreYy .marker.cross{stroke:#333333;}#mermaid-svg-lzCFF8NlsyxNreYy svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-lzCFF8NlsyxNreYy .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-lzCFF8NlsyxNreYy .cluster-label text{fill:#333;}#mermaid-svg-lzCFF8NlsyxNreYy .cluster-label span{color:#333;}#mermaid-svg-lzCFF8NlsyxNreYy .label text,#mermaid-svg-lzCFF8NlsyxNreYy span{fill:#333;color:#333;}#mermaid-svg-lzCFF8NlsyxNreYy .node rect,#mermaid-svg-lzCFF8NlsyxNreYy .node circle,#mermaid-svg-lzCFF8NlsyxNreYy .node ellipse,#mermaid-svg-lzCFF8NlsyxNreYy .node polygon,#mermaid-svg-lzCFF8NlsyxNreYy .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-lzCFF8NlsyxNreYy .node .label{text-align:center;}#mermaid-svg-lzCFF8NlsyxNreYy .node.clickable{cursor:pointer;}#mermaid-svg-lzCFF8NlsyxNreYy .arrowheadPath{fill:#333333;}#mermaid-svg-lzCFF8NlsyxNreYy .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-lzCFF8NlsyxNreYy .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-lzCFF8NlsyxNreYy .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-lzCFF8NlsyxNreYy .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-lzCFF8NlsyxNreYy .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-lzCFF8NlsyxNreYy .cluster text{fill:#333;}#mermaid-svg-lzCFF8NlsyxNreYy .cluster span{color:#333;}#mermaid-svg-lzCFF8NlsyxNreYy div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-lzCFF8NlsyxNreYy :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
-
客户端发起一个写操作请求
-
Leader服务器将客户端的请求转化为事务Proposal提案,同时为每个Proposal分配一个全局的ID,即zxid
-
Leader服务器为每个Follower服务器分配一个单独的队列,然后将需要广播的Proposal依次放到队列中去,并且根据FIFO策略进行消息发送
-
Follower接收到Proposal后,会首先将其以事务日志的方式写入本地磁盘中,写入成功后向Leader反馈一个Ack响应消息
-
Leader接收到超过半数以上Follower的Ack响应消息后,即认为消息发送成功,可以发送commit消息
-
Leaderl向所有Follower广播commit消息,同时自身也会完成事务提交。Follower接收到commit消息后,会将上一条事务提交
此图对应的是上图的流程图
崩溃恢复
#mermaid-svg-UsLx4IY7O4G7AOKa {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-UsLx4IY7O4G7AOKa .error-icon{fill:#552222;}#mermaid-svg-UsLx4IY7O4G7AOKa .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-UsLx4IY7O4G7AOKa .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-UsLx4IY7O4G7AOKa .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-UsLx4IY7O4G7AOKa .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-UsLx4IY7O4G7AOKa .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-UsLx4IY7O4G7AOKa .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-UsLx4IY7O4G7AOKa .marker{fill:#333333;stroke:#333333;}#mermaid-svg-UsLx4IY7O4G7AOKa .marker.cross{stroke:#333333;}#mermaid-svg-UsLx4IY7O4G7AOKa svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-UsLx4IY7O4G7AOKa .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-UsLx4IY7O4G7AOKa .cluster-label text{fill:#333;}#mermaid-svg-UsLx4IY7O4G7AOKa .cluster-label span{color:#333;}#mermaid-svg-UsLx4IY7O4G7AOKa .label text,#mermaid-svg-UsLx4IY7O4G7AOKa span{fill:#333;color:#333;}#mermaid-svg-UsLx4IY7O4G7AOKa .node rect,#mermaid-svg-UsLx4IY7O4G7AOKa .node circle,#mermaid-svg-UsLx4IY7O4G7AOKa .node ellipse,#mermaid-svg-UsLx4IY7O4G7AOKa .node polygon,#mermaid-svg-UsLx4IY7O4G7AOKa .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-UsLx4IY7O4G7AOKa .node .label{text-align:center;}#mermaid-svg-UsLx4IY7O4G7AOKa .node.clickable{cursor:pointer;}#mermaid-svg-UsLx4IY7O4G7AOKa .arrowheadPath{fill:#333333;}#mermaid-svg-UsLx4IY7O4G7AOKa .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-UsLx4IY7O4G7AOKa .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-UsLx4IY7O4G7AOKa .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-UsLx4IY7O4G7AOKa .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-UsLx4IY7O4G7AOKa .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-UsLx4IY7O4G7AOKa .cluster text{fill:#333;}#mermaid-svg-UsLx4IY7O4G7AOKa .cluster span{color:#333;}#mermaid-svg-UsLx4IY7O4G7AOKa div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-UsLx4IY7O4G7AOKa :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
Zab协议崩溃恢复同时满足的要求
-
已经产生提交(Proposal)的提案(过半数都Ack),Follower必须执行
-
未提交的提案直接丢弃 - 未proposal给follower的提案
Leader挂,重新选举
#mermaid-svg-7uEHZouC5uoJZf6x {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-7uEHZouC5uoJZf6x .error-icon{fill:#552222;}#mermaid-svg-7uEHZouC5uoJZf6x .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-7uEHZouC5uoJZf6x .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-7uEHZouC5uoJZf6x .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-7uEHZouC5uoJZf6x .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-7uEHZouC5uoJZf6x .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-7uEHZouC5uoJZf6x .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-7uEHZouC5uoJZf6x .marker{fill:#333333;stroke:#333333;}#mermaid-svg-7uEHZouC5uoJZf6x .marker.cross{stroke:#333333;}#mermaid-svg-7uEHZouC5uoJZf6x svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-7uEHZouC5uoJZf6x .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-7uEHZouC5uoJZf6x .cluster-label text{fill:#333;}#mermaid-svg-7uEHZouC5uoJZf6x .cluster-label span{color:#333;}#mermaid-svg-7uEHZouC5uoJZf6x .label text,#mermaid-svg-7uEHZouC5uoJZf6x span{fill:#333;color:#333;}#mermaid-svg-7uEHZouC5uoJZf6x .node rect,#mermaid-svg-7uEHZouC5uoJZf6x .node circle,#mermaid-svg-7uEHZouC5uoJZf6x .node ellipse,#mermaid-svg-7uEHZouC5uoJZf6x .node polygon,#mermaid-svg-7uEHZouC5uoJZf6x .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-7uEHZouC5uoJZf6x .node .label{text-align:center;}#mermaid-svg-7uEHZouC5uoJZf6x .node.clickable{cursor:pointer;}#mermaid-svg-7uEHZouC5uoJZf6x .arrowheadPath{fill:#333333;}#mermaid-svg-7uEHZouC5uoJZf6x .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-7uEHZouC5uoJZf6x .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-7uEHZouC5uoJZf6x .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-7uEHZouC5uoJZf6x .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-7uEHZouC5uoJZf6x .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-7uEHZouC5uoJZf6x .cluster text{fill:#333;}#mermaid-svg-7uEHZouC5uoJZf6x .cluster span{color:#333;}#mermaid-svg-7uEHZouC5uoJZf6x div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-7uEHZouC5uoJZf6x :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
新Leader满足要求
-
新Leader必须都是己经提交了Proposall的Follower服务器节点
-
新选举的Leader节点中含有最大的zxid)这样做的好处是可以避免Leader服务器检查Proposal的提交和丢弃工作
数据恢复
#mermaid-svg-qMKmG0f2qbJMwHhQ {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-qMKmG0f2qbJMwHhQ .error-icon{fill:#552222;}#mermaid-svg-qMKmG0f2qbJMwHhQ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-qMKmG0f2qbJMwHhQ .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-qMKmG0f2qbJMwHhQ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-qMKmG0f2qbJMwHhQ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-qMKmG0f2qbJMwHhQ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-qMKmG0f2qbJMwHhQ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-qMKmG0f2qbJMwHhQ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-qMKmG0f2qbJMwHhQ .marker.cross{stroke:#333333;}#mermaid-svg-qMKmG0f2qbJMwHhQ svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-qMKmG0f2qbJMwHhQ .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-qMKmG0f2qbJMwHhQ .cluster-label text{fill:#333;}#mermaid-svg-qMKmG0f2qbJMwHhQ .cluster-label span{color:#333;}#mermaid-svg-qMKmG0f2qbJMwHhQ .label text,#mermaid-svg-qMKmG0f2qbJMwHhQ span{fill:#333;color:#333;}#mermaid-svg-qMKmG0f2qbJMwHhQ .node rect,#mermaid-svg-qMKmG0f2qbJMwHhQ .node circle,#mermaid-svg-qMKmG0f2qbJMwHhQ .node ellipse,#mermaid-svg-qMKmG0f2qbJMwHhQ .node polygon,#mermaid-svg-qMKmG0f2qbJMwHhQ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-qMKmG0f2qbJMwHhQ .node .label{text-align:center;}#mermaid-svg-qMKmG0f2qbJMwHhQ .node.clickable{cursor:pointer;}#mermaid-svg-qMKmG0f2qbJMwHhQ .arrowheadPath{fill:#333333;}#mermaid-svg-qMKmG0f2qbJMwHhQ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-qMKmG0f2qbJMwHhQ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-qMKmG0f2qbJMwHhQ .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-qMKmG0f2qbJMwHhQ .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-qMKmG0f2qbJMwHhQ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-qMKmG0f2qbJMwHhQ .cluster text{fill:#333;}#mermaid-svg-qMKmG0f2qbJMwHhQ .cluster span{color:#333;}#mermaid-svg-qMKmG0f2qbJMwHhQ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-qMKmG0f2qbJMwHhQ :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
数据恢复
新Leader选举
开始接收客户端请求前,首先确认事务日志的所有过半同意的提案已经commit
某个Follower同步完成Leader的提案后,才会将其加入到真正可用的Follower列表中
CAP理伦
分布式系统最多同时满足CAP其中的两项,不可能三项同时满足
Zookeeper:满足的是CP的两项要求
#mermaid-svg-mhxTgXOp8Fpks4s8 {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-mhxTgXOp8Fpks4s8 .error-icon{fill:#552222;}#mermaid-svg-mhxTgXOp8Fpks4s8 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-mhxTgXOp8Fpks4s8 .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-mhxTgXOp8Fpks4s8 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-mhxTgXOp8Fpks4s8 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-mhxTgXOp8Fpks4s8 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-mhxTgXOp8Fpks4s8 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-mhxTgXOp8Fpks4s8 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-mhxTgXOp8Fpks4s8 .marker.cross{stroke:#333333;}#mermaid-svg-mhxTgXOp8Fpks4s8 svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-mhxTgXOp8Fpks4s8 .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-mhxTgXOp8Fpks4s8 .cluster-label text{fill:#333;}#mermaid-svg-mhxTgXOp8Fpks4s8 .cluster-label span{color:#333;}#mermaid-svg-mhxTgXOp8Fpks4s8 .label text,#mermaid-svg-mhxTgXOp8Fpks4s8 span{fill:#333;color:#333;}#mermaid-svg-mhxTgXOp8Fpks4s8 .node rect,#mermaid-svg-mhxTgXOp8Fpks4s8 .node circle,#mermaid-svg-mhxTgXOp8Fpks4s8 .node ellipse,#mermaid-svg-mhxTgXOp8Fpks4s8 .node polygon,#mermaid-svg-mhxTgXOp8Fpks4s8 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-mhxTgXOp8Fpks4s8 .node .label{text-align:center;}#mermaid-svg-mhxTgXOp8Fpks4s8 .node.clickable{cursor:pointer;}#mermaid-svg-mhxTgXOp8Fpks4s8 .arrowheadPath{fill:#333333;}#mermaid-svg-mhxTgXOp8Fpks4s8 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-mhxTgXOp8Fpks4s8 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-mhxTgXOp8Fpks4s8 .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-mhxTgXOp8Fpks4s8 .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-mhxTgXOp8Fpks4s8 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-mhxTgXOp8Fpks4s8 .cluster text{fill:#333;}#mermaid-svg-mhxTgXOp8Fpks4s8 .cluster span{color:#333;}#mermaid-svg-mhxTgXOp8Fpks4s8 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-mhxTgXOp8Fpks4s8 :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
CAP
一致性(Consistency):集群之间的数据保持一致
可用性(Available):系统服务一直可用,且保证用户每个操作能在有限时间返回结果
分区容错性(Partition Tolerance):遇到网络分区故障,仍然能对外提供可用服务
#mermaid-svg-C1sDLTtuoUy5qssH {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-C1sDLTtuoUy5qssH .error-icon{fill:#552222;}#mermaid-svg-C1sDLTtuoUy5qssH .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-C1sDLTtuoUy5qssH .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-C1sDLTtuoUy5qssH .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-C1sDLTtuoUy5qssH .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-C1sDLTtuoUy5qssH .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-C1sDLTtuoUy5qssH .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-C1sDLTtuoUy5qssH .marker{fill:#333333;stroke:#333333;}#mermaid-svg-C1sDLTtuoUy5qssH .marker.cross{stroke:#333333;}#mermaid-svg-C1sDLTtuoUy5qssH svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-C1sDLTtuoUy5qssH .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-C1sDLTtuoUy5qssH .cluster-label text{fill:#333;}#mermaid-svg-C1sDLTtuoUy5qssH .cluster-label span{color:#333;}#mermaid-svg-C1sDLTtuoUy5qssH .label text,#mermaid-svg-C1sDLTtuoUy5qssH span{fill:#333;color:#333;}#mermaid-svg-C1sDLTtuoUy5qssH .node rect,#mermaid-svg-C1sDLTtuoUy5qssH .node circle,#mermaid-svg-C1sDLTtuoUy5qssH .node ellipse,#mermaid-svg-C1sDLTtuoUy5qssH .node polygon,#mermaid-svg-C1sDLTtuoUy5qssH .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-C1sDLTtuoUy5qssH .node .label{text-align:center;}#mermaid-svg-C1sDLTtuoUy5qssH .node.clickable{cursor:pointer;}#mermaid-svg-C1sDLTtuoUy5qssH .arrowheadPath{fill:#333333;}#mermaid-svg-C1sDLTtuoUy5qssH .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-C1sDLTtuoUy5qssH .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-C1sDLTtuoUy5qssH .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-C1sDLTtuoUy5qssH .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-C1sDLTtuoUy5qssH .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-C1sDLTtuoUy5qssH .cluster text{fill:#333;}#mermaid-svg-C1sDLTtuoUy5qssH .cluster span{color:#333;}#mermaid-svg-C1sDLTtuoUy5qssH div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-C1sDLTtuoUy5qssH :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
为什么Zookeeper不满足A
极端情况下,服务端会丢弃一些客户端请求
进行Leader选举时,集群不可用
脚本
集群统一启动、关闭、状态查看脚本
zk.sh
#!/bin/bash
# 运行此脚本前必须把当前机器人的公私密钥给到目标运行机器 == 要不然每次运行此脚本时都会叫你输入每台目标机器的密码
# 命令1(本机生成RSA公私密钥):ssh-keygen -t rsa
# 命令2(将密钥传给目标三台机器即192.168.19.107、192.168.19.108、192.168.19.109 ):ssh-copy-id root@目标机器IP
for currentHostName in 192.168.19.107 192.168.19.108 192.168.19.109
do
echo "=================zookeeper【${currentHostName}】【$1】==============================="
case $1 in
"start") {
ssh $currentHostName "cd /opt/module/zookeeper-3.9.1 && sh bin/zkServer.sh start"
}
;;
"stop") {
ssh $currentHostName "cd /opt/module/zookeeper-3.9.1 && sh bin/zkServer.sh stop"
}
;;
"status") {
ssh $currentHostName "cd /opt/module/zookeeper-3.9.1 && sh bin/zkServer.sh status"
}
;;
*) {
echo "未知命令,仅支持start|stop|status"
}
esac
done
源码分析(粗略)
辅助源码
持久化
数据存储: 集群中的数据会在内存(树)、磁盘中各存一份
接口: 快照【org.apache.zookeeper.server.persistence.SnapShot】、事务记录【org.apache.zookeeper.server.persistence.TxnLog】
事务日志(txnlog): ZooKeeper会将所有的写操作以事务的形式记录在事务日志中,这些写操作包括创建节点、更新节点数据、删除节点等。事务日志是一个追加写的日志文件,用于记录每个写操作的详细信息。通过事务日志,ZooKeeper可以保证数据的一致性和持久性
快照(snapshot): ZooKeeper定期会生成一个快照文件,用于保存当前内存中所有节点的状态。快照文件包含了所有节点的数据和元数据信息。当ZooKeeper服务器启动时,会首先加载最新的快照文件,然后通过回放事务日志来恢复到最新的状态。
Zookeeper启动数据恢复流程: 先加载最新的快照文件,然后通过回放事务日志来将数据恢复到最新的状态
序列化
接口: 序列化、反序列化【org.apache.jute.Record】
服务端启动流程
入口类: org.apache.zookeeper.server.quorum.QuorumPeerMain#main
服务端选举Leader流程
Leader、Follower数据同步流程
概括: Follower必须去看Leader保持一致,而不是Leader跟Follower保持一致
【Follower】Learner: org.apache.zookeeper.server.quorum.Learner#registerWithLeader
【Leader】LearnerHandler: org.apache.zookeeper.server.quorum.LearnerHandler#run
#mermaid-svg-PtD2ilnxs8EpAFOt {font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt .error-icon{fill:#552222;}#mermaid-svg-PtD2ilnxs8EpAFOt .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-PtD2ilnxs8EpAFOt .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-PtD2ilnxs8EpAFOt .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-PtD2ilnxs8EpAFOt .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-PtD2ilnxs8EpAFOt .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-PtD2ilnxs8EpAFOt .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-PtD2ilnxs8EpAFOt .marker{fill:#333333;stroke:#333333;}#mermaid-svg-PtD2ilnxs8EpAFOt .marker.cross{stroke:#333333;}#mermaid-svg-PtD2ilnxs8EpAFOt svg{font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-PtD2ilnxs8EpAFOt .label{font-family:“trebuchet ms”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt .cluster-label text{fill:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt .cluster-label span{color:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt .label text,#mermaid-svg-PtD2ilnxs8EpAFOt span{fill:#333;color:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt .node rect,#mermaid-svg-PtD2ilnxs8EpAFOt .node circle,#mermaid-svg-PtD2ilnxs8EpAFOt .node ellipse,#mermaid-svg-PtD2ilnxs8EpAFOt .node polygon,#mermaid-svg-PtD2ilnxs8EpAFOt .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-PtD2ilnxs8EpAFOt .node .label{text-align:center;}#mermaid-svg-PtD2ilnxs8EpAFOt .node.clickable{cursor:pointer;}#mermaid-svg-PtD2ilnxs8EpAFOt .arrowheadPath{fill:#333333;}#mermaid-svg-PtD2ilnxs8EpAFOt .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-PtD2ilnxs8EpAFOt .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-PtD2ilnxs8EpAFOt .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-PtD2ilnxs8EpAFOt .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-PtD2ilnxs8EpAFOt .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-PtD2ilnxs8EpAFOt .cluster text{fill:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt .cluster span{color:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-PtD2ilnxs8EpAFOt :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
差异化同步
回滚同步
提交同步
全量同步
同步方式
Diff:Leader、Follower都一样,无需任何操作
Trunc Follower:Follower的事务ID比Leader大,则Follower回滚成跟Leader一致
既有适合小白学习的零基础资料,也有适合3年以上经验的小伙伴深入学习提升的进阶课程,涵盖了95%以上大数据知识点,真正体系化!
由于文件比较多,这里只是将部分目录截图出来,全套包含大厂面经、学习笔记、源码讲义、实战项目、大纲路线、讲解视频,并且后续会持续更新
color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-PtD2ilnxs8EpAFOt .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-PtD2ilnxs8EpAFOt .cluster text{fill:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt .cluster span{color:#333;}#mermaid-svg-PtD2ilnxs8EpAFOt div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:“trebuchet ms”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-PtD2ilnxs8EpAFOt :root{–mermaid-font-family:“trebuchet ms”,verdana,arial,sans-serif;}
差异化同步
回滚同步
提交同步
全量同步
同步方式
Diff:Leader、Follower都一样,无需任何操作
Trunc Follower:Follower的事务ID比Leader大,则Follower回滚成跟Leader一致
[外链图片转存中…(img-i4YoJ5OH-1714807691530)]
[外链图片转存中…(img-3Kfg8XLR-1714807691530)]
[外链图片转存中…(img-VGIgdcja-1714807691530)]
既有适合小白学习的零基础资料,也有适合3年以上经验的小伙伴深入学习提升的进阶课程,涵盖了95%以上大数据知识点,真正体系化!
由于文件比较多,这里只是将部分目录截图出来,全套包含大厂面经、学习笔记、源码讲义、实战项目、大纲路线、讲解视频,并且后续会持续更新