apache-pulsar-2.10.0集群安装

apache-pulsar-2.10.0集群安装

​ 本文将展示如何apache-pulsar-2.10.0集群

1. Pulsar简介

1.1 Pulsar 的关键特性

• Pulsar 的单个实例原生支持多个集群，可跨机房在集群间无缝地完成消息复制。
• 极低的发布延迟和端到端延迟。
• 可无缝扩展到超过一百万个 topic。
• 简单的客户端 API，支持 Java、Go、Python 和 C++。
• 支持多种 topic 订阅模式（独占订阅、共享订阅、故障转移订阅）。
• 通过 Apache BookKeeper 提供的持久化消息存储机制保证消息传递 。
• 由轻量级的 serverless 计算框架 Pulsar Functions 实现流原生的数据处理。
• 基于 Pulsar Functions 的 serverless connector 框架 Pulsar IO 使得数据更易移入、移出 Apache Pulsar。
• 分层式存储可在数据陈旧时，将数据从热存储卸载到冷/长期存储（如S3、GCS）中。

官方文档地址：https://pulsar.apache.org/docs/zh-CN/concepts-overview/

1.2 Pulsar集群组成

• ZooKeeper 集群（3(或多) 个 ZooKeeper 节点组成）
• bookie 集群（也称为 BookKeeper 集群，3(或多) 个 BookKeeper 节点组成）
• broker 集群（3(或多) 个 Pulsar 节点组成）

说明：Pulsar 的安装包已包含了搭建集群所需的各个组件库。本文后续内容基于Pulsar包自带的组件进行；

2. 集群安装

2.1 安装准备

• JDK版本不低于 JDK 8

• 下载最新版本pulsar，官方下载地址：https://pulsar.apache.org/zh-CN/download/
  自行下载并上传至服务器，
  或者直接下载：
  wget https://archive.apache.org/dist/pulsar/pulsar-2.10.0/apache-pulsar-2.10.0-bin.tar.gz

• 解压到指定目录

  tar -zxvf apache-pulsar-2.10.0-bin.tar.gz -C /home/bigdata/apps
  ln -s apache-pulsar-2.10.0 pulsar

• 指定三台机器，ip示例如下：
  192.168.1.193
  192.168.1.194
  192.168.1.195

• 当前操作用户为：bigdata

2.2 Zookeeper集群安装

2.2.1 Zookeeper配置文件修改

#在192.168.1.193, 192.168.1.194, 192.168.1.195上分别创建目录
mkdir -pv /home/bigdata/data/zookeeper
mkdir -pv /home/bigdata/dataLog/zookeeper

#192.168.1.193上执行
echo 1 > /home/bigdata/data/zookeeper/myid 
#192.168.1.194上执行
echo 2 > /home/bigdata/data/zookeeper/myid 
#192.168.1.195上执行
echo 3 > /home/bigdata/data/zookeeper/myid

#分别修改三台zookeeper配置
vim /home/bigdata/apps/pulsar/conf/zookeeper.conf

三台机器上zookeeper配置如下：

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/bigdata/data/zookeeper
dataLogDir=/home/bigdata/dataLog/zookeeper
clientPort=12181
admin.enableServer=true
admin.serverPort=9990
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
forceSync=yes
sslQuorum=false
portUnification=false
metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
metricsProvider.httpPort=8000
metricsProvider.exportJvmInfo=true
server.1=192.168.1.193:12888:13888
server.2=192.168.1.194:12888:13888
server.3=192.168.1.195:12888:13888

2.2.2 Zookeeper集群启动

#分别启动三台zookeeper: 执行后台运行命令
./bin/pulsar-daemon start zookeeper

#停止zookeeper命令
./bin/pulsar-daemon stop zookeeper

#查看启动情况
ps aux|grep pulsar

#查看端口启动情况
netstat -tpnl |grep 12181

2.2.3 验证ZooKeeper节点是否启动成功

#执行 zookeeper 客户端连接命令
./bin/pulsar zookeeper-shell -server 127.0.0.1:12181
或
./bin/pulsar zookeeper-shell -server 192.168.1.193:12181
./bin/pulsar zookeeper-shell -server 192.168.1.194:12181
./bin/pulsar zookeeper-shell -server 192.168.1.195:12181

#查看所有zk节点数据,命令如下
ls /

#显示如下,表示zk的集群已经搭建好了
[zookeeper]

2.2.4 初始化Zookeeper集群元数据

#先查看端口占用情况,避免后面启动时会报端口占用错误
netstat -tpnl |grep 8080  
netstat -tpnl |grep 8443
netstat -tpnl |grep 6650
netstat -tpnl |grep 6651

# 在任一个 zooKeeper 节点,如:192.168.1.193，初始化集群元数据
# 进入Apache-pulsar 目录
# 执行命令初始化集群元数据
./bin/pulsar initialize-cluster-metadata \
--cluster yx-pulsar-cluster-geo \
--zookeeper 192.168.1.193:12181,192.168.1.194:12181,192.168.1.195:12181 \
--configuration-store 192.168.1.193:12181,192.168.1.194:12181,192.168.1.195:12181 \
--web-service-url http://192.168.1.193:18080,192.168.1.194:18080,192.168.1.195:18080 \
--web-service-url-tls https://192.168.1.193:8443,192.168.1.194:8443,192.168.1.195:8443 \
--broker-service-url pulsar://192.168.1.193:6650,192.168.1.194:6650,192.168.1.195:6650 \
--broker-service-url-tls pulsar+ssl://192.168.1.193:6651,192.168.1.194:6651,192.168.1.195:6651

#连接任意一台zookeeper,比如:192.168.1.193
./bin/pulsar zookeeper-shell -server 127.0.0.1:12181
#查询数据
ls /
#显示如下,有bookies,pulsar等信息,表示成功初始化
[admin, bookies, ledgers, pulsar, stream, zookeeper]

2.3 BookKeeper集群安装

2.3.1 BookKeeper配置文件修改

#分别每个节点执行

#创建bookie所需要目录
mkdir -pv /home/bigdata/data/bookkeeper/journal
mkdir -pv /home/bigdata/data/bookkeeper/ledger

#注意下面的配置，修改每个节点对应的ip
# 进入bookie 配置文件目录，编辑 bookkeeper.conf 文件
vim bookkeeper.conf

以192.168.1.193为例，说明bookkeeper.conf配置文件的内容

# advertisedAddress 修改为服务器对应的ip,在另外两台服务器也做对应的修改advertisedAddress=192.168.1.193
advertisedAddress=192.168.1.193

# 修改以下两个文件目录地址
journalDirectory=/home/bigdata/data/bookkeeper/journal
ledgerDirectories=/home/bigdata/data/bookkeeper/ledger

# 修改zk地址和端口信息
zkServers=192.168.1.193:12181,192.168.1.194:12181,192.168.1.195:12181

#httpServerPort默认也是8000,建议修改,我这里修改为:8100
prometheusStatsHttpPort=8100

2.3.2 初始化Bookkeeper集群元数据

# 执行初始化元数据命令；若出现提示，输入 Y，继续（只需在任意一个bookie节点执行一次）
./bin/bookkeeper shell metaformat

# 响应
2022-06-14T17:38:15,661+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2022-06-14T17:38:15,661+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.version=3.10.0-1062.el7.x86_64
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.name=bigdata
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/bigdata
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/bigdata/apps/apache-pulsar-2.10.0
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=1898MB
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=2048MB
2022-06-14T17:38:15,662+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=2048MB
2022-06-14T17:38:15,672+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=192.168.1.193:12181,192.168.1.194:12181,192.168.1.195:12181 sessionTimeout=30000 watcher=org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase@68f4865
2022-06-14T17:38:15,695+0800 [main] INFO  org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2022-06-14T17:38:15,705+0800 [main] INFO  org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 1048575 Bytes
2022-06-14T17:38:15,733+0800 [main] INFO  org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
2022-06-14T17:38:15,767+0800 [main-SendThread(192.168.1.193:12181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server centos7-pulsar-c21/192.168.1.193:12181.
2022-06-14T17:38:15,768+0800 [main-SendThread(192.168.1.193:12181)] INFO  org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to authenticate using SASL (unknown error)
2022-06-14T17:38:15,780+0800 [main-SendThread(192.168.1.193:12181)] INFO  org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /192.168.1.195:39384, server: centos7-pulsar-c21/192.168.1.193:12181
2022-06-14T17:38:15,799+0800 [main-SendThread(192.168.1.193:12181)] INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server centos7-pulsar-c21/192.168.1.193:12181, session id = 0x10001dd180e0001, negotiated timeout = 30000
2022-06-14T17:38:15,808+0800 [main-EventThread] INFO  org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now.
Ledger root already exists. Are you sure to format bookkeeper metadata? This may cause data loss. (Y or N) Y
2022-06-14T17:38:41,069+0800 [main] INFO  org.apache.bookkeeper.discover.ZKRegistrationManager - Successfully formatted BookKeeper metadata
2022-06-14T17:38:41,185+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x10001dd180e0001 closed
2022-06-14T17:38:41,185+0800 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x10001dd180e0001

看到“Successfully formatted BookKeeper metadata”表明成功！

2.3.3 BookKeeper集群启动

各个节点配置好后，分别执行以下命令启动BookKeeper：

#启动bookie命令,以后台进程启动bookie
./bin/pulsar-daemon start bookie

bookie关闭命令如下：

./bin/pulsar-daemon stop bookie

验证是否启动成功：

./bin/bookkeeper shell bookiesanity

# 响应
2022-06-14T17:40:59,217+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2022-06-14T17:40:59,217+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
2022-06-14T17:40:59,217+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.version=3.10.0-1062.el7.x86_64
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.name=bigdata
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/bigdata
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/home/bigdata/apps/apache-pulsar-2.10.0
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=1890MB
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=2048MB
2022-06-14T17:40:59,218+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=2048MB
2022-06-14T17:40:59,226+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=192.168.1.193:12181,192.168.1.194:12181,192.168.1.195:12181 sessionTimeout=30000 watcher=org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase@71e9ebae
2022-06-14T17:40:59,252+0800 [main] INFO  org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2022-06-14T17:40:59,261+0800 [main] INFO  org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 1048575 Bytes
2022-06-14T17:40:59,289+0800 [main] INFO  org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
2022-06-14T17:40:59,328+0800 [main-SendThread(192.168.1.195:12181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server centos7-pulsar-c23/192.168.1.195:12181.
2022-06-14T17:40:59,328+0800 [main-SendThread(192.168.1.195:12181)] INFO  org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to authenticate using SASL (unknown error)
2022-06-14T17:40:59,343+0800 [main-SendThread(192.168.1.195:12181)] INFO  org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /192.168.1.195:39832, server: centos7-pulsar-c23/192.168.1.195:12181
2022-06-14T17:40:59,361+0800 [main-SendThread(192.168.1.195:12181)] INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server centos7-pulsar-c23/192.168.1.195:12181, session id = 0x30001d8e81e0007, negotiated timeout = 30000
2022-06-14T17:40:59,371+0800 [main-EventThread] INFO  org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now.
2022-06-14T17:41:01,246+0800 [main] INFO  org.apache.bookkeeper.client.BookKeeper - Weighted ledger placement is not enabled
2022-06-14T17:41:01,385+0800 [main-EventThread] INFO  org.apache.bookkeeper.discover.ZKRegistrationClient - Update BookieInfoCache (writable bookie) 192.168.1.193:3181 -> BookieServiceInfo{properties={}, endpoints=[EndpointInfo{id=bookie, port=3181, host=192.168.1.193, protocol=bookie-rpc, auth=[], extensions=[]}]}
2022-06-14T17:41:01,386+0800 [main-EventThread] INFO  org.apache.bookkeeper.discover.ZKRegistrationClient - Update BookieInfoCache (writable bookie) 192.168.1.194:3181 -> BookieServiceInfo{properties={}, endpoints=[EndpointInfo{id=bookie, port=3181, host=192.168.1.194, protocol=bookie-rpc, auth=[], extensions=[]}]}
2022-06-14T17:41:01,386+0800 [main-EventThread] INFO  org.apache.bookkeeper.discover.ZKRegistrationClient - Update BookieInfoCache (writable bookie) 192.168.1.195:3181 -> BookieServiceInfo{properties={}, endpoints=[EndpointInfo{id=bookie, port=3181, host=192.168.1.195, protocol=bookie-rpc, auth=[], extensions=[]}]}
2022-06-14T17:41:01,577+0800 [main-EventThread] INFO  org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [192.168.1.195:3181] for ledger: 0
2022-06-14T17:41:01,579+0800 [main] INFO  org.apache.bookkeeper.tools.cli.commands.bookie.SanityTestCommand - Create ledger 0
2022-06-14T17:41:02,089+0800 [bookkeeper-io-3-1] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - Successfully connected to bookie: 192.168.1.195:3181 [id: 0xa41f5d28, L:/192.168.1.195:47680 - R:192.168.1.195/192.168.1.195:3181]
2022-06-14T17:41:02,127+0800 [bookkeeper-io-3-1] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - connection [id: 0xa41f5d28, L:/192.168.1.195:47680 - R:192.168.1.195/192.168.1.195:3181] authenticated as BookKeeperPrincipal{ANONYMOUS}
2022-06-14T17:41:02,336+0800 [main] INFO  org.apache.bookkeeper.tools.cli.commands.bookie.SanityTestCommand - Written 10 entries in ledger 0
2022-06-14T17:41:02,434+0800 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  org.apache.bookkeeper.client.ReadOnlyLedgerHandle - Closing recovered ledger 0 at entry 9
2022-06-14T17:41:02,480+0800 [main] INFO  org.apache.bookkeeper.tools.cli.commands.bookie.SanityTestCommand - Read 10 entries from ledger 0
2022-06-14T17:41:02,518+0800 [main] INFO  org.apache.bookkeeper.tools.cli.commands.bookie.SanityTestCommand - Deleted ledger 0
2022-06-14T17:41:02,518+0800 [main] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - Closing the per channel bookie client for 192.168.1.195:3181
2022-06-14T17:41:02,525+0800 [main] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - Closing the per channel bookie client for 192.168.1.195:3181
2022-06-14T17:41:02,525+0800 [bookkeeper-io-3-1] INFO  org.apache.bookkeeper.proto.PerChannelBookieClient - Disconnected from bookie channel [id: 0xa41f5d28, L:/192.168.1.195:47680 ! R:192.168.1.195/192.168.1.195:3181]
2022-06-14T17:41:04,583+0800 [main] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x30001d8e81e0007 closed
2022-06-14T17:41:04,583+0800 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x30001d8e81e0007
2022-06-14T17:41:04,583+0800 [main] INFO  org.apache.bookkeeper.tools.cli.commands.bookie.SanityTestCommand - Bookie sanity test succeeded

看到“Bookie sanity test succeeded”表明成功！

2.4 Pulsar的Broker集群安装

2.4.1 Broker配置文件修改

分别每个节点执行

vim conf/broker.conf

以192.168.1.193为例，说明broker.conf配置文件的内容

# pulsar cluster名，和 ZooKeeper 里初始化元数据时指定的集群名(--cluster yx-pulsar-cluster-geo)相同
clusterName=yx-pulsar-cluster-geo

# 元数据存储URL
metadataStoreUrl=192.168.1.193:12181,192.168.1.194:12181,192.168.1.195:12181

# 配置数据的元数据存储URL，可以为空，为空将使用metadataStoreUrl
configurationMetadataStoreUrl=192.168.1.193:12181,192.168.1.194:12181,192.168.1.195:12181

# 对外用于查找当前机器的机器名或IP地址，如果不配置，会使用InetAddress.getLocalHost().getHostName()
advertisedAddress=192.168.1.193

# http请求服务端口
webServicePort=18080

2.4.2 Broker集群启动

各个节点配置好后，分别执行以下命令启动Broker：

# 以后台进程启动 broker
./bin/pulsar-daemon start broker

broker关闭命令如下：

./bin/pulsar-daemon stop broker

2.4.3 验证Broker集群是否启动成功

分别每个节点执行

vim conf/client.conf

以192.168.1.193为例，说明.conf配置文件的内容

webServiceUrl=http://localhost:18080/

执行以下命令，查看yx-pulsar-cluster-geo集群 brokers 节点情况

./bin/pulsar-admin brokers list yx-pulsar-cluster-geo
或
./bin/pulsar-admin --admin-url http://192.168.1.193:18080 brokers list yx-pulsar-cluster-geo

显示如下:表式集群搭建成功：

$ ./bin/pulsar-admin --admin-url http://192.168.1.193:18080 brokers list yx-pulsar-cluster-geo
192.168.1.194:18080
192.168.1.195:18080
192.168.1.193:18080

2.4.4 消息收发测试

消费者订阅

#在yx subscription 订阅中 consume 一条消息到topic:yx-tech的命令
./bin/pulsar-client consume yx-tech -s "yx subscription"

开启另一个终端发送数据，观察订阅者收到数据：content:hello-yx

#生产者发送: 向名称为 my-topic 的 topic 发送一条简单的消息 hello-pulsar
#--messages后面双引号中的内容，若以英文逗号分隔，则消息会分开发送
./bin/pulsar-client produce yx-tech --messages "hello-yx"