Kafka Topic Replication Factor Expansion

Kafka Topic Replication Factor Expansion

The current test-topic has a replication factor of 1 and exists only on Broker 2. Now, the replication factor needs to be increased to 3 (i.e., add 2 more replicas).

1.Ensure the Kafka cluster has at least 3 brokers

Run the following command to check the available brokers:

/opt/kafka/bin/zookeeper-shell.sh 192.168.126.104:2181 ls /brokers/ids

2.Create a partition reassignment plan (JSON file)

We need to manually specify the new replica assignment. For example, assuming the broker list is [1, 2, 3], we can distribute the replicas of test-topic across [1, 2, 3].

Create the increase-replication-factor.json file:

{
  "version": 1,
  "partitions": [
    {
      "topic": "test-topic",
      "partition": 0,
      "replicas": [1, 2, 3]  // New replica distribution (Leader is usually the first one)
    }
  ]
}

Explanation:
replicas: [1, 2, 3] means:

• Leader: Broker 1 (the first broker in the list is usually the leader)

• Followers: Broker 2 and Broker 3

3.Execute the replica reassignment

Run the following command to apply the new replica assignment:

/opt/kafka/bin/kafka-reassign-partitions.sh \
  --bootstrap-server 192.168.126.104:9092 \
  --reassignment-json-file increase-replication-factor.json \
  --execute

Output:

[2025-04-05 15:44:50,208] WARN [AdminClient clientId=reassign-partitions-tool] The DescribeTopicPartitions API is not supported, using Metadata API to describe topics. (org.apache.kafka.clients.admin.KafkaAdminClient)

Current partition replica assignment

{"version":1,"partitions":[{"topic":"test-topic","partition":0,"replicas":[2],"log_dirs":["any"]}]}

Save this to use as the --reassignment-json-file option during rollback

4.Check the reassignment progress

/opt/kafka/bin/kafka-reassign-partitions.sh \
  --bootstrap-server 192.168.126.104:9092 \
  --reassignment-json-file increase-replication-factor.json \
  --verify

Output:

[2025-04-05 15:52:03,277] WARN [AdminClient clientId=reassign-partitions-tool] The DescribeTopicPartitions API is not supported, using Metadata API to describe topics. (org.apache.kafka.clients.admin.KafkaAdminClient)

Status of partition reassignment:
Reassignment of partition test-topic-0 is completed.

Clearing broker-level throttles on brokers 1,2,3
Clearing topic-level throttles on topic test-topic

5.Verify that the replication factor has been updated to 3

You can use the kafka-topics.sh --describe command to confirm the updated replication factor.

/opt/kafka/bin/kafka-topics.sh \
  --describe \
  --bootstrap-server 192.168.126.104:9092 \
  --topic test-topic

Output:

[2025-04-05 15:53:26,815] WARN [AdminClient clientId=adminclient-1] The DescribeTopicPartitions API is not supported, using Metadata API to describe topics. (org.apache.kafka.clients.admin.KafkaAdminClient)
Topic: test-topic       TopicId: h_YjbwfaTPOUJPMccYoXoA PartitionCount: 1       ReplicationFactor: 3    Configs: segment.bytes=1073741824
        Topic: test-topic       Partition: 0    Leader: 2       Replicas: 1,2,3 Isr: 2,3,1      Elr: N/A        LastKnownElr: N/A

6.Possible Issues

6.1 LEADER_NOT_AVAILABLE or NOT_ENOUGH_REPLICAS

• Check whether Brokers 1, 2, and 3 are all running properly:

/opt/kafka/bin/kafka-broker-api-versions.sh --bootstrap-server 192.168.126.104:9092

• If any broker is down, the replica reassignment will fail. You need to fix the broker and then retry the reassignment.

• Replica Sync Delay (Incomplete ISR)

  • If the ISR (In-Sync Replicas) only shows [1, 2] and does not include 3, it means that Broker 3 has not completed synchronization yet. You can wait for a while or check Broker 3’s logs for more details.