Apache Pulsar命令行操作tenants租户、namespace命名空间、topic

1. 命令行操作tenants租户

1.1 查询租户列表

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin tenants list
"public"
"pulsar"
[root@pulsar1 apache-pulsar-2.9.1]#

1.2 创建租户

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin tenants create my-tenant
[root@pulsar1 apache-pulsar-2.9.1]#

1.3 查询租户配置信息

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin tenants get my-tenant
{
  "adminRoles" : [ ],
  "allowedClusters" : [ "pulsar-cluster" ]
}
[root@pulsar1 apache-pulsar-2.9.1]#

1.4 更新租户配置信息

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin tenants update -c pulsar-cluster my-tenant
[root@pulsar1 apache-pulsar-2.9.1]#

1.5 删除租户

如果租户下面有namespace，则不能删除租户

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin tenants delete my-tenant
[root@pulsar1 apache-pulsar-2.9.1]#

2. namespace

2.1 namespace的相关概念

namespace下的topic会自动继承namespace的属性

namespace分为两种：

1. 本地namespace：仅对定义它的集群可见。
2. 全局namespace：跨集群可见，这依赖于是否在namespace中设置了跨集群拷贝数据的功能

2.2 namespace的命令行操作

2.2.1 创建namespace

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces create my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

2.2.2 查看某个租户下的namespace列表

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces list my-tenant
"my-tenant/my-ns"
[root@pulsar1 apache-pulsar-2.9.1]#

2.2.3 删除namespace

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces delete my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

2.2.4 获取namespace的policies

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces policies my-tenant/my-ns
{
  "auth_policies" : {
    "namespace_auth" : { },
    "destination_auth" : { },
    "subscription_auth_roles" : { }
  },
  "replication_clusters" : [ "pulsar-cluster" ],
  "bundles" : {
    "boundaries" : [ "0x00000000", "0x40000000", "0x80000000", "0xc0000000", "0xffffffff" ],
    "numBundles" : 4
  },
  "backlog_quota_map" : { },
  "clusterDispatchRate" : { },
  "topicDispatchRate" : { },
  "subscriptionDispatchRate" : { },
  "replicatorDispatchRate" : { },
  "clusterSubscribeRate" : { },
  "publishMaxMessageRate" : { },
  "latency_stats_sample_rate" : { },
  "deleted" : false,
  "encryption_required" : false,
  "subscription_auth_mode" : "None",
  "offload_threshold" : -1,
  "schema_auto_update_compatibility_strategy" : "Full",
  "schema_compatibility_strategy" : "UNDEFINED",
  "is_allow_auto_update_schema" : true,
  "schema_validation_enforced" : false,
  "subscription_types_enabled" : [ ],
  "properties" : { }
}
[root@pulsar1 apache-pulsar-2.9.1]#

2.2.5 配置多个集群之间的数据复制

语法如下：

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces get-clusters my-tenant/my-ns
"pulsar-cluster"
[root@pulsar1 apache-pulsar-2.9.1]#
[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces set-clusters my-tenant/my-ns --clusters pulsar-cluster2
[root@pulsar1 apache-pulsar-2.9.1]#

2.2.6 namespace的quota设置

查看quota配置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces get-backlog-quotas my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

进行quota设置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces set-backlog-quota --limit 10G --limitTime 36000 --policy consumer_backlog_eviction my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

参数说明如下：

• –limit：Bookeeper使用磁盘空间大小
• –limitTime：单位为秒，在该时间内，Bookeeper使用磁盘空间大小
• –policy：有3个可选值
  1. producer_request_hold：Broker暂停运行，不再接收produce新的请求
  2. producer_exception：Broker抛出异常，和客户端断开连接
  3. consumer_backlog_eviction：Broker丢弃老的消息

删除quota设置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces remove-backlog-quota my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

2.2.7 持久化的配置

查看持久化配置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces get-persistence my-tenant/my-ns
null
[root@pulsar1 apache-pulsar-2.9.1]# 

进行持久化设置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces set-persistence --bookkeeper-ack-quorum 2 --bookkeeper-ensemble 3 --bookkeeper-write-quorum 2 --ml-mark-delete-max-rate 0 my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

各参数含义如下：

• –bookkeeper-ack-quorum：一条entry消息需要多少个Bookeeper确认，才算produce成功。默认0
• –bookkeeper-ensemble：一个topic能使用的Bookeeper数量。默认0
• –bookkeeper-write-quorum：一条entry消息要写入多少个Bookeeper。默认0
• –ml-mark-delete-max-rate：mark-delete操作的最大速率。默认0.0

2.2.8 消息存活时间TTL设置

获取TTL设置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces get-message-ttl my-tenant/my-ns
null
[root@pulsar1 apache-pulsar-2.9.1]#

进行TTL设置，单位为秒

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces set-message-ttl --messageTTL 600 my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

删除TTL设置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces remove-message-ttl my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

2.2.9 Topic的消息发送速率

获取Topic的消息发送速率配置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces get-dispatch-rate my-tenant/my-ns
null
[root@pulsar1 apache-pulsar-2.9.1]#

进行Topic的消息发送速率配置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces set-dispatch-rate --dispatch-rate-period 2 --msg-dispatch-rate 2000 --byte-dispatch-rate 1048576 my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

参数说明如下：

• –dispatch-rate-period：进行限制的时间大小，默认为1秒
• –msg-dispatch-rate：在dispatch-rate-period时间范围内，能接收的消息数量。默认-1
• –byte-dispatch-rate：在dispatch-rate-period时间范围内，能接收的数据字节大小。默认-1

2.2.10 Topic的消息接收速率

获取Topic的消息接收速率配置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces get-subscription-dispatch-rate my-tenant/my-ns
null
[root@pulsar1 apache-pulsar-2.9.1]#

进行Topic的消息接收速率配置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces set-subscription-dispatch-rate --dispatch-rate-period 2 --msg-dispatch-rate 2000 --byte-dispatch-rate 1048576 my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

参数说明如下：

• –dispatch-rate-period：进行限制的时间大小，默认为1秒
• –msg-dispatch-rate：在dispatch-rate-period时间范围内，能接收的消息数量。默认-1
• –byte-dispatch-rate：在dispatch-rate-period时间范围内，能接收的数据字节大小。默认-1

2.2.11 Topic的多个集群之间的复制速率

获取Topic的多个集群之间的复制速率配置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces get-replicator-dispatch-rate my-tenant/my-ns
null
[root@pulsar1 apache-pulsar-2.9.1]#

进行Topic的多个集群之间的复制速率配置

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin namespaces set-replicator-dispatch-rate --dispatch-rate-period 2 --msg-dispatch-rate 2000 --byte-dispatch-rate 1048576 my-tenant/my-ns
[root@pulsar1 apache-pulsar-2.9.1]#

参数说明如下：

• –dispatch-rate-period：进行限制的时间大小，默认为1秒
• –msg-dispatch-rate：在dispatch-rate-period时间范围内，能接收的消息数量。默认-1
• –byte-dispatch-rate：在dispatch-rate-period时间范围内，能接收的数据字节大小。默认-1

3. Topic

3.1 Topic的总类

1. 持久化Topic：消息会被保存到Bookeeper，命名格式为：persistent://tenant/namespace/topic

2. 非持久化Topic：消息不会被保存到Bookeeper，命名格式为：non-persistent://tenant/namespace/topic

3.2 topic的命令行操作

3.2.1 创建Topic，和列出namespace下所有topic

1. 创建无分区的topic

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin topics create non-persistent://my-tenant/my-ns/my-non-partitioned-topic
[root@pulsar1 apache-pulsar-2.9.1]#

2. 创建有分区的topic，并列出namespace下的所有topic

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin topics create-partitioned-topic persistent://my-tenant/my-ns/my-partitioned-topic --partitions 3
[root@pulsar1 apache-pulsar-2.9.1]#
[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin topics list my-tenant/my-ns
"persistent://my-tenant/my-ns/my-partitioned-topic-partition-0"
"persistent://my-tenant/my-ns/my-partitioned-topic-partition-1"
"persistent://my-tenant/my-ns/my-partitioned-topic-partition-2"
[root@pulsar1 apache-pulsar-2.9.1]#

可以看到my-non-partitioned-topic这个topic已经被删除了。这是因为一个topic被创建后，如果在创建后的60秒内不活动，就会被自动删除。可以通过如下两个参数进行控制

• Brokerdeleteinactivetopicsenabenabled：表示是否启动自动删除，默认值为true
• BrokerDeleteInactiveTopicsFrequencySeconds：表示不活动的时间，默认为60s

3.2.2 更新topic

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin topics update-partitioned-topic persistent://my-tenant/my-ns/my-partitioned-topic --partitions 6
[root@pulsar1 apache-pulsar-2.9.1]# 

3.2.3 删除topic

1. 删除无分区topic

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin topics delete non-persistent://my-tenant/my-ns/my-non-partitioned-topic
[root@pulsar1 apache-pulsar-2.9.1]#

2. 删除有分区topic

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin topics delete-partitioned-topic persistent://my-tenant/my-ns/my-partitioned-topic
[root@pulsar1 apache-pulsar-2.9.1]#

3.2.4 列出topic下的subscriptions

[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin topics subscriptions persistent://public/default/my-partitioned-topic
"consume-test"
[root@pulsar1 apache-pulsar-2.9.1]#

3.2.4 删除topic下的subscription

需要该subscription没有active consumer

[root@pulsar1 apache-pulsar-2.9.1]# 
[root@pulsar1 apache-pulsar-2.9.1]# bin/pulsar-admin topics unsubscribe -s "consume-test"  persistent://public/my-partitioned-topic
[root@pulsar1 apache-pulsar-2.9.1]#