kafka中节点如何服役和退役？

一、服役新节点

1.新节点准备：增加一个新的服务器/虚拟机

2.执行负载均衡操作

1)创建一个要均衡的主题

        创建一个文件：vi topics-to-move.json
        写上如下代码，如果多个topic 可以使用,分隔
        {
          "topics": [
            {"topic": "first"}
          ],
          "version": 1
        }

2）生成一个负载均衡的计划

        bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --topics-to-move-json-file         topics-to-move.json --broker-list "0,1,2,3" --generate

将未来的分区策略拷贝一份

        {"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[3,2,0],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[0,3,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[1,0,2],"log_dirs":["any","any","any"]}]}

3）创建副本存储计划（所有副本存储在broker0、broker1、broker2、broker3中）

        编写json： vi increase-replication-factor.json

        内容：{"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[3,2,0],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[0,3,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[1,0,2],"log_dirs":["any","any","any"]}]}
以上这个内容来自于第二步的执行计划。

4）执行副本存储计划

        bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --reassignment-json-file         increase-replication-factor.json --execute

5）验证副本存储计划

        bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --reassignment-json-file         increase-replication-factor.json --verify

二、退役旧节点

1.执行负载均衡操作

1)创建一个要均衡的主题

        创建一个文件：vi topics-to-move.json
        写上如下代码，如果多个topic 可以使用,分隔
        {
          "topics": [
            {"topic": "first"}
          ],
          "version": 1
        }

2）生成一个负载均衡的计划

        bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --topics-to-move-json-file         topics-to-move.json --broker-list "0,1,2" --generate

3）创建副本存储计划（所有副本存储在broker0、broker1、broker2中）

        编写json： vi increase-replication-factor.json

        内容：{"version":1,"partitions":[{"topic":"first","partition":0,"replicas":[0,2,1],"log_dirs":["any","any","any"]},{"topic":"first","partition":1,"replicas":[1,0,2],"log_dirs":["any","any","any"]},{"topic":"first","partition":2,"replicas":[2,1,0],"log_dirs":["any","any","any"]}]}
以上这个内容来自于第二步的执行计划。

4）执行副本存储计划

        bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --reassignment-json-file         increase-replication-factor.json --execute

5）验证副本存储计划

        bin/kafka-reassign-partitions.sh --bootstrap-server hadoop11:9092 --reassignment-json-file         increase-replication-factor.json --verify

2.执行停止命令

        bin/kafka-server-stop.sh