kafka最好用的脚本一：kafka-topic

kafka-topic 使用手册

官方说明:

Create, delete, describe, or change a topic.

Option                                   Description
------                                   -----------
--alter                                  Alter the number of partitions,
                                           replica assignment, and/or
                                           configuration for the topic.
--config <String: name=value>            A topic configuration override for the
                                           topic being created or altered.The
                                           following is a list of valid
                                           configurations:
                                         	cleanup.policy
                                         	compression.type
                                         	delete.retention.ms
                                         	file.delete.delay.ms
                                         	flush.messages
                                         	flush.ms
                                         	follower.replication.throttled.
                                           replicas
                                         	index.interval.bytes
                                         	leader.replication.throttled.replicas
                                         	max.message.bytes
                                         	message.format.version
                                         	message.timestamp.difference.max.ms
                                         	message.timestamp.type
                                         	min.cleanable.dirty.ratio
                                         	min.compaction.lag.ms
                                         	min.insync.replicas
                                         	preallocate
                                         	retention.bytes
                                         	retention.ms
                                         	segment.bytes
                                         	segment.index.bytes
                                         	segment.jitter.ms
                                         	segment.ms
                                         	unclean.leader.election.enable
                                         See the Kafka documentation for full
                                           details on the topic configs.
--create                                 Create a new topic.
--delete                                 Delete a topic
--delete-config <String: name>           A topic configuration override to be
                                           removed for an existing topic (see
                                           the list of configurations under the
                                           --config option).
--describe                               List details for the given topics.
--disable-rack-aware                     Disable rack aware replica assignment
--force                                  Suppress console prompts
--help                                   Print usage information.
--if-exists                              if set when altering or deleting
                                           topics, the action will only execute
                                           if the topic exists
--if-not-exists                          if set when creating topics, the
                                           action will only execute if the
                                           topic does not already exist
--list                                   List all available topics.
--partitions <Integer: # of partitions>  The number of partitions for the topic
                                           being created or altered (WARNING:
                                           If partitions are increased for a
                                           topic that has a key, the partition
                                           logic or ordering of the messages
                                           will be affected
--replica-assignment <String:            A list of manual partition-to-broker
  broker_id_for_part1_replica1 :           assignments for the topic being
  broker_id_for_part1_replica2 ,           created or altered.
  broker_id_for_part2_replica1 :
  broker_id_for_part2_replica2 , ...>
--replication-factor <Integer:           The replication factor for each
  replication factor>                      partition in the topic being created.
--topic <String: topic>                  The topic to be create, alter or
                                           describe. Can also accept a regular
                                           expression except for --create option
--topics-with-overrides                  if set when describing topics, only
                                           show topics that have overridden
                                           configs
--unavailable-partitions                 if set when describing topics, only
                                           show partitions whose leader is not
                                           available
--under-replicated-partitions            if set when describing topics, only
                                           show under replicated partitions
--zookeeper <String: urls>               REQUIRED: The connection string for
                                           the zookeeper connection in the form
                                           host:port. Multiple URLS can be
                                           given to allow fail-over.
复制代码

使用方法:

可以使用该脚本进行topic的创建、删除、查看和配置修改。

• 1、topic创建

> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic topic_name
> bin/kafka-topics.sh --create --zookeeper localhost:2181/kafka  --replication-factor 3 --partitions 2 --topic topic_name (zk地址需要写完整)

> 此外，凡是在 Kafka documentation 中列出的topic的配置项都可以通过 --config 字段对默认值进行覆盖。
复制代码

• 2、topic删除

> Kafka-topics.sh --zookeeper <zookeeper.connect> --delete --topic topic_name (删除指定的topic)
复制代码

• 3、topic查看

> kafka-topic.sh --zookeeper <zookeeper.connect> --list / --describe （查看kafka集群中所有topic）

> kafka-topic.sh --zookeeper <zookeeper.connect> --describe --under-replicated-partitions（查看正在同步的分区（可能正在同步，也可能发生异常））

> kafka-topic.sh --zookeeper <zookeeper.connect> --describe --unavalible-partitions（查看没有leader的分区）

> kafka-topic.sh --zookeeper <zookeeper.connect> --describe --topics-with-overrides（查看一下覆盖了默认配置的分区）

> kafka-topic.sh --zookeeper <zookeeper.connect> --describe --topic topic_name（查看指定topic的相关信息）
复制代码

• 4、topic config修改

> bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic my-topic --partitions 1 --replication-factor 1 --config max.message.bytes=64000 --config flush.messages=1 (可以在topic创建时进行参数的指定)

> bin/kafka-topics.sh --zookeeper localhost:2181 --alter --config retention.ms=0 --topic topic_name (对已经存在的topic进行config的修改)

复制代码