一、服役新节点
1.新节点准备:增加一个新的服务器/虚拟机
2.执行负载均衡操作
1)创建一个要均衡的主题
创建一个文件:vi topics-to-move.json
写上如下代码,如果多个topic 可以使用,分隔
{
"topics": [
{"topic": "first"}
],
"version": 1
}
2)生成一个负载均衡的计划
bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --topics-to-move-json-file topics-to-move.json --broker-list "0,1,2,3" --generate
将未来的分区策略拷贝一份
{"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[3,2,0],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[0,3,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[1,0,2],"log_dirs":["any","any","any"]}]}
3)创建副本存储计划(所有副本存储在broker0、broker1、broker2、broker3中)
编写json: vi increase-replication-factor.json
内容:{"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[3,2,0],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[0,3,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[1,0,2],"log_dirs":["any","any","any"]}]}
以上这个内容来自于第二步的执行计划。
4)执行副本存储计划
bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --reassignment-json-file increase-replication-factor.json --execute
5)验证副本存储计划
bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --reassignment-json-file increase-replication-factor.json --verify
二、退役旧节点
1.执行负载均衡操作
1)创建一个要均衡的主题
创建一个文件:vi topics-to-move.json
写上如下代码,如果多个topic 可以使用,分隔
{
"topics": [
{"topic": "first"}
],
"version": 1
}
2)生成一个负载均衡的计划
bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --topics-to-move-json-file topics-to-move.json --broker-list "0,1,2" --generate
3)创建副本存储计划(所有副本存储在broker0、broker1、broker2中)
编写json: vi increase-replication-factor.json
内容:{"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[0,2,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[1,0,2],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[2,1,0],"log_dirs":["any","any","any"]}]}
以上这个内容来自于第二步的执行计划。
4)执行副本存储计划
bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --reassignment-json-file increase-replication-factor.json --execute
5)验证副本存储计划
bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --reassignment-json-file increase-replication-factor.json --verify
2.执行停止命令
bin/kafka-server-stop.sh