几周前,我写了一篇文章,解释了如何使用Kubernetes创建Neo4j因果集群,并且…我想弄清楚如何模拟网络分区,该分区将使领导者处于少数席位并迫使选举。
我们已经使用iptables命令在AWS的内部工具上完成了此操作,但是不幸的是,在我的容器中没有该容器,该容器仅具有BusyBox提供的实用程序。
幸运的是,其中之一是route命令,它将使我们能够实现相同的目的。
回顾一下,我有3个Neo4j Pod已启动并正在运行:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
neo4j-0 1/1 Running 0 6h
neo4j-1 1/1 Running 0 6h
neo4j-2 1/1 Running 0 6h
我们可以检查route命令是否可用:
$ kubectl exec neo4j-0 -- ls -alh /sbin/route
lrwxrwxrwx 1 root root 12 Oct 18 18:58 /sbin/route -> /bin/busybox
让我们看一下每个服务器当前扮演的角色:
$ kubectl exec neo4j-0 -- bin/cypher-shell "CALL dbms.cluster.role()"
role
"FOLLOWER"
Bye!
$ kubectl exec neo4j-1 -- bin/cypher-shell "CALL dbms.cluster.role()"
role
"FOLLOWER"
Bye!
$ kubectl exec neo4j-2 -- bin/cypher-shell "CALL dbms.cluster.role()"
role
"LEADER"
Bye!
稍微说一下:我可以在没有用户名和密码的情况下调用cypher-shell ,因为我通过将以下内容放入conf / neo4j.conf来禁用授权:
dbms.connector.bolt.enabled=true
回到网络分区……我们需要通过运行以下命令从其他两台服务器中划分neo4j-2 :
$ kubectl exec neo4j-2 -- route add -host neo4j-0.neo4j.default.svc.cluster.local reject && \
kubectl exec neo4j-2 -- route add -host neo4j-1.neo4j.default.svc.cluster.local reject && \
kubectl exec neo4j-0 -- route add -host neo4j-2.neo4j.default.svc.cluster.local reject && \
kubectl exec neo4j-1 -- route add -host neo4j-2.neo4j.default.svc.cluster.local reject
如果我们查看neo4j-2的日志,可以看到它与其他两台服务器断开连接后已退出:
$ kubectl exec neo4j-2 -- cat logs/debug.log
...
2016-12-04 11:30:10.186+0000 INFO [o.n.c.c.c.RaftMachine] Moving to FOLLOWER state after not receiving heartbeat responses in this election timeout period. Heartbeats received: []
...
谁被接任领导?
$ kubectl exec neo4j-0 -- bin/cypher-shell "CALL dbms.cluster.role()"
role
"LEADER"
Bye!
$ kubectl exec neo4j-1 -- bin/cypher-shell "CALL dbms.cluster.role()"
role
"FOLLOWER"
Bye!
$ kubectl exec neo4j-2 -- bin/cypher-shell "CALL dbms.cluster.role()"
role
"FOLLOWER"
Bye!
看起来像neo4j-0! 让我们将一些数据放入数据库中:
$ kubectl exec neo4j-0 -- bin/cypher-shell "CREATE (:Person {name: 'Mark'})"
Added 1 nodes, Set 1 properties, Added 1 labels
Bye!
让我们检查该节点是否到达其他两个服务器。 我们希望它在neo4j-1上,而不在neo4j-2上:
$ kubectl exec neo4j-1 -- bin/cypher-shell "MATCH (p:Person) RETURN p"
p
(:Person {name: "Mark"})
Bye!
$ kubectl exec neo4j-2 -- bin/cypher-shell "MATCH (p:Person) RETURN p"
Bye!
在neo4j-2上,随着选举超时触发器的出现,我们会在日志中反复看到这些类型的条目,但无法获得对其发出的投票请求的任何响应:
$ kubectl exec neo4j-2 -- cat logs/debug.log
...
2016-12-04 11:32:56.735+0000 INFO [o.n.c.c.c.RaftMachine] Election timeout triggered
2016-12-04 11:32:56.736+0000 INFO [o.n.c.c.c.RaftMachine] Election started with vote request: Vote.Request from MemberId{ca9b954c} {term=11521, candidate=MemberId{ca9b954c}, lastAppended=68, lastLogTerm=11467} and members: [MemberId{484178c4}, MemberId{0acdb8dd}, MemberId{ca9b954c}]
...
我们通过查看raft-messages.log可以看到那些投票请求,可以通过在conf / neo4j.conf中设置以下属性来启用:
causal_clustering.raft_messages_log_enable=true
$ kubectl exec neo4j-2 -- cat logs/raft-messages.log
...
11:33:42.101 -->MemberId{484178c4}: Request: Vote.Request from MemberId{ca9b954c} {term=11537, candidate=MemberId{ca9b954c}, lastAppended=68, lastLogTerm=11467}
11:33:42.102 -->MemberId{0acdb8dd}: Request: Vote.Request from MemberId{ca9b954c} {term=11537, candidate=MemberId{ca9b954c}, lastAppended=68, lastLogTerm=11467}
11:33:45.432 -->MemberId{484178c4}: Request: Vote.Request from MemberId{ca9b954c} {term=11538, candidate=MemberId{ca9b954c}, lastAppended=68, lastLogTerm=11467}
11:33:45.433 -->MemberId{0acdb8dd}: Request: Vote.Request from MemberId{ca9b954c} {term=11538, candidate=MemberId{ca9b954c}, lastAppended=68, lastLogTerm=11467}
11:33:48.362 -->MemberId{484178c4}: Request: Vote.Request from MemberId{ca9b954c} {term=11539, candidate=MemberId{ca9b954c}, lastAppended=68, lastLogTerm=11467}
11:33:48.362 -->MemberId{0acdb8dd}: Request: Vote.Request from MemberId{ca9b954c} {term=11539, candidate=MemberId{ca9b954c}, lastAppended=68, lastLogTerm=11467}
...
要“修复”网络分区,我们只需要删除我们之前运行的所有命令:
$ kubectl exec neo4j-2 -- route delete neo4j-0.neo4j.default.svc.cluster.local reject && \
kubectl exec neo4j-2 -- route delete neo4j-1.neo4j.default.svc.cluster.local reject && \
kubectl exec neo4j-0 -- route delete neo4j-2.neo4j.default.svc.cluster.local reject && \
kubectl exec neo4j-1 -- route delete neo4j-2.neo4j.default.svc.cluster.local reject
现在,让我们检查一下neo4j-2现在是否具有我们之前创建的节点:
$ kubectl exec neo4j-2 -- bin/cypher-shell "MATCH (p:Person) RETURN p"
p
(:Person {name: "Mark"})
Bye!
目前为止就这样了!
翻译自: https://www.javacodegeeks.com/2016/12/kubernetes-simulating-network-partition.html