pulsar版本升级

已于 2022-08-19 10:28:24 修改
阅读量1.3k 收藏 1
点赞数 1
文章标签： 运维
于 2022-08-11 15:10:47 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_45112997/article/details/126285860
版权

pulsar升级

当前版本
  • 2.7.1
升级到的版本
  • 本次升级至2.9.3，pulsar当前最新版本为2.10.1
  • https://pulsar.apache.org/release-notes/versioned/pulsar-2.9.3
升级方式
  • 滚动升级
升级顺序
  1. Upgrade ZooKeeper (optional)
  2. Upgrade bookies
  3. Upgrade brokers

升级步骤

一、准备包

1、下载2.9.3版本的tar包
  • https://archive.apache.org/dist/pulsar/pulsar-2.9.3/
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/apache-pulsar-2.9.3-bin.tar.gz
2、下载2.9.3版本的内置连接器
  • https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/
 #原内置连接器
 pulsar-io-canal-2.7.1.nar
 pulsar-io-elastic-search-2.7.1.nar
 pulsar-io-file-2.7.1.nar
 pulsar-io-hdfs2-2.7.1.nar
 pulsar-io-hdfs3-2.7.1.nar
 pulsar-io-influxdb-2.7.1.nar
 pulsar-io-jdbc-mariadb-2.7.1.nar
 pulsar-io-jdbc-postgres-2.7.1.nar
 pulsar-io-kafka-2.7.1.nar
 pulsar-io-kafka-connect-adaptor-2.7.1.nar
 pulsar-io-netty-2.7.1.nar
 pulsar-io-rabbitmq-2.7.1.nar
 pulsar-io-redis-2.7.1.nar
 
tar xf apache-pulsar-2.9.3-bin.tar.gz -C /opt
cd /opt/apache-pulsar-2.9.3
mkdir connectors
cd connectors
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-canal-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-elastic-search-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-file-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-hdfs2-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-hdfs3-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-influxdb-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-jdbc-mariadb-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-jdbc-postgres-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-kafka-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-kafka-connect-adaptor-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-netty-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-rabbitmq-2.9.3.nar
wget https://archive.apache.org/dist/pulsar/pulsar-2.9.3/connectors/pulsar-io-redis-2.9.3.nar
3、下载kop
  • https://github.com/streamnative/kop/releases
wget https://github.com/streamnative/kop/releases/download/v2.9.3.4/pulsar-protocol-handler-kafka-2.9.3.4.nar
mkdir protocol-handler
mv pulsar-protocol-handler-kafka-2.9.3.4.nar protocol-handler

二、准备配置文件

1、同步2.7.1版本的配置文件到新版本目录下
cp /opt/pulsar/conf/* /opt/apache-pulsar-2.9.3/conf/
2、修改bookeeper.conf

  • Zookeeper占用8000端口

    ZooKeeper 端口
Pulsar 默认使用的 ZooKeeper 配置文件中 metrics 端口设成了 8000，而非 ZooKeeper 默认的 7000。ZooKeeper 和 BooKkeeper metrics 端口冲突。

这里确实会存在和 BookKeeper metric 端口冲突的问题，如果是在同一台机器上混布 BookKeeper 和 Zookeeper 的话，建议修改其中一个端口。
可以通过 export PULSAR_EXTRA_OPTS=-Dstats_server_port=8002 修改 ZooKeeper metrics 端口号。

PULSAR_EXTRA_OPTS="-Dstats_server_port=8001" pulsar-daemon start zookeeper

    如果启动zookeeper时，指定其metrics为非8000端口，则忽略修改bookeeper端口

#按报错提示修改配置
cat bookeeper.conf
prometheusStatsHttpPort=8000
advertisedAddress=

可选：sed -i 's/prometheusStatsHttpPort=8000/prometheusStatsHttpPort=8001/g' /opt/apache-pulsar-2.9.3/conf/bookkeeper.conf
sed -i 's/advertisedAddress=/advertisedAddress=10.11.0.12/g' /opt/apache-pulsar-2.9.3/conf/bookkeeper.conf

三、升级

  • https://pulsar.apache.org/docs/2.9.3/administration-upgrade
1、升级zookeeper（可选）

本次升级为必选 否则启动broker时报错：https://github.com/apache/pulsar/pull/11198

升级 ZooKeeper 服务器的步骤如下：

1. 终止 ZooKeeper服务。
2. 升级二进制文件和配置文件。
3. 用新的二进制文件启动 ZooKeeper 服务。
4. 用 pulsar zookeeper-shell 连接新的 ZooKeeper 服务，并运行命令测试其运行状态。
5. 运行ZooKeeper服务几天，观察 ZooKeeper 集群的运行状态，确保可以正常运行。
1.1、停止旧版本zookeeper
cd /opt/pulsar
./bin/pulsar-daemon stop zookeeper
1.2、升级二进制文件和配置文件。
1.3、用新的二进制文件启动 ZooKeeper 服务
cd /opt/apache-pulsar-2.9.3
./bin/pulsar-daemon start zookeeper
1.4、升级后验证
用 pulsar zookeeper-shell 连接新的 ZooKeeper 服务，并运行命令测试其运行状态。
#输出
WatchedEvent state:SyncConnected type:None path:null
[zk: xxxx:6181(CONNECTED) 0] ls /
[admin, bookies, counters, ledgers, loadbalance, managed-ledgers, namespace, schemas, stream, zookeeper
]
失败回滚
关闭出故障的 ZooKeeper 节点，使用旧版本的二进制文件和配置文件重新启动 ZooKeeper。
2、升级bookeeper

  • https://bookkeeper.apache.org/docs/admin/upgrade/

  • https://pulsar.apache.org/docs/2.9.3/administration-upgrade

  • 升级前，需要确定是否一次性升级整个集群，包括停机升级和滚动升级。

  • 在滚动升级中，一次升级一个 bookie。 在停机升级中，需先关闭全部集群，升级所有 bookie 后再启动集群。

2.1、禁用 autorecovery
cd /opt/pulsar
./bin/bookkeeper shell autorecovery -disable
2.2、停止旧的bookeeper
cd /opt/pulsar
./bin/pulsar-daemon stop bookie
2.3、新版本只读模式启动 bookie，以验证此新版本 bookie 的读性能
cd /opt/apache-pulsar-2.9.3
./bin/pulsar bookie --readOnly
2.4、报错及解决
报错
15:03:02.872 [main] ERROR org.apache.bookkeeper.server.Main - Failed to build bookie server
java.net.UnknownHostException: Trying to listen on loopback address, 127.0.0.1:3181 but this is forbidden by default (see ServerConfiguration#getAllowLoopback()).
If this happen, you can consider specifying the network interface to listen on (e.g. listeningInterface=eth0) or specifying the advertised address (e.g. advertisedAddress=172.x.y.z)
	at org.apache.bookkeeper.bookie.Bookie.getBookieAddress(Bookie.java:608) ~[org.apache.bookkeeper-bookkeeper-server-4.14.2.jar:4.14.2]
	at org.apache.bookkeeper.bookie.Bookie.getBookieId(Bookie.java:563) ~[org.apache.bookkeeper-bookkeeper-server-4.14.2.jar:4.14.2]
	at org.apache.bookkeeper.proto.BookieNettyServer.<init>(BookieNettyServer.java:166) ~[org.apache.bookkeeper-bookkeeper-server-4.14.2.jar:4.14.2]
	at org.apache.bookkeeper.proto.BookieServer.<init>(BookieServer.java:118) ~[org.apache.bookkeeper-bookkeeper-server-4.14.2.jar:4.14.2]
	at org.apache.bookkeeper.server.service.BookieService.<init>(BookieService.java:52) ~[org.apache.bookkeeper-bookkeeper-server-4.14.2.jar:4.14.2]
	at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:304) ~[org.apache.bookkeeper-bookkeeper-server-4.14.2.jar:4.14.2]
	at org.apache.bookkeeper.server.Main.doMain(Main.java:226) [org.apache.bookkeeper-bookkeeper-server-4.14.2.jar:4.14.2]
	at org.apache.bookkeeper.server.Main.main(Main.java:208) [org.apache.bookkeeper-bookkeeper-server-4.14.2.jar:4.14.2]
	

 [main] ERROR org.apache.bookkeeper.common.component.AbstractLifecycleComponent - Failed to start Component: stats-provider
java.lang.RuntimeException: java.io.IOException: Failed to bind to /0.0.0.0:8000
解决
#按报错提示修改配置
#涉及到的配置：
conf/bookeeper.conf
prometheusStatsHttpPort=8000
httpServerPort=8000
advertisedAddress=
报错
  • ./bin/bookkeeper shell decommissionbookie
14:29:58.801 [main] ERROR org.apache.bookkeeper.client.BookKeeperAdmin - Autorecovery is disabled. So giving up!
14:29:58.801 [main] ERROR org.apache.bookkeeper.tools.cli.commands.bookies.DecommissionCommand - Received exception in DecommissionBookieCmd
org.apache.bookkeeper.replication.ReplicationException$UnavailableException: Autorecovery is disabled. So giving up!
	at org.apache.bookkeeper.client.BookKeeperAdmin.triggerAudit(BookKeeperAdmin.java:1523) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.client.BookKeeperAdmin.decommissionBookie(BookKeeperAdmin.java:1565) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.tools.cli.commands.bookies.DecommissionCommand.decommission(DecommissionCommand.java:97) [org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.tools.cli.commands.bookies.DecommissionCommand.apply(DecommissionCommand.java:82) [org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.bookie.BookieShell$DecommissionBookieCmd.runCmd(BookieShell.java:1956) [org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.bookie.BookieShell$MyCommand.runCmd(BookieShell.java:238) [org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.bookie.BookieShell.run(BookieShell.java:2278) [org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
解决
 ./bin/bookkeeper shell autorecovery -enable
报错
13:56:45.817 [main] ERROR org.apache.bookkeeper.bookie.Bookie - There are directories without a cookie, and this is neither a new environment, nor is storage expansion enabled. Empty directories are [/data/pulsar/data/bookkeeper/journal/current, /data/pulsar/data/bookkeeper/ledgers/current]
13:56:45.817 [main] INFO  org.apache.bookkeeper.proto.BookieNettyServer - Shutting down BookieNettyServer
13:56:45.836 [main] ERROR org.apache.bookkeeper.server.Main - Failed to build bookie server
org.apache.bookkeeper.bookie.BookieException$InvalidCookieException:
	at org.apache.bookkeeper.bookie.Bookie.checkEnvironmentWithStorageExpansion(Bookie.java:494) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.bookie.Bookie.checkEnvironment(Bookie.java:273) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.bookie.Bookie.<init>(Bookie.java:731) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:152) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.proto.BookieServer.<init>(BookieServer.java:120) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.server.service.BookieService.<init>(BookieService.java:52) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:304) ~[org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.server.Main.doMain(Main.java:226) [org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
	at org.apache.bookkeeper.server.Main.main(Main.java:208) [org.apache.bookkeeper-bookkeeper-server-4.14.5.jar:4.14.5]
解决
  • https://github.com/apache/pulsar/issues/3121
 run decommissionbookie to decommission the bookie first.
 The decommission command will re-replicate the data and delete the cookie in the zookeeper. 
 So the bookie can restart freshly
 运行decommissionbookie停用 bookie。
 decommission 命令将重新复制数据并删除 zookeeper 中的 cookie。
 所以 bookie 可以重新启动
 
 ./bin/bookkeeper shell autorecovery -enable
 ./bin/bookkeeper shell decommissionbookie
 ./bin/pulsar-daemon start bookie
 ./bin/bookkeeper shell recover xxxx:9181
 验证查看bookie可用节点个数
 ./bin/bookkeeper shell listbookies -rw
2.5、测试通过，后台启动bookie
cd /opt/apache-pulsar-2.9.3
./bin/pulsar-daemon start bookie

#验证是否启动成功
jps -l
#查看端口
ss -anptl |grep 进程号
ps -ef |grep 进程号

./bin/bookkeeper shell simpletest --ensemble 3 --writeQuorum 3 --ackQuorum 3 --numEntries 3

./bin/bookkeeper shell bookiesanity
#启动成功输出
- Bookie sanity test succeeded
2.6、升级所有节点的 bookie 后，重新启用 autorecovery
cd /opt/apache-pulsar-2.9.3
./bin/bookkeeper shell autorecovery -enable
失败回滚
  • https://bookkeeper.apache.org/docs/admin/autorecovery
方式一、启动一个新节点
 ./bin/bookkeeper shell autorecovery -enable
 ./bin/bookkeeper shell decommissionbookie
 ./bin/pulsar-daemon start bookie
 ./bin/bookkeeper shell recover xxxx:9181
 验证查看bookie可用节点个数
 ./bin/bookkeeper shell listbookies -rw
#手动 
e.g:
./bin/bookkeeper shell recover xxxx:3181
#自动：
停止出故障的 bookie 节点，通过 autorecovery 用其他集群中的 bookie 来替换出现异常的 bookie 节点。
也可以手动执行以下命令恢复
./bin/bookkeeper autorecovery
3、升级broker

  1. 终止 broker。

    cd /opt/pulsar
./bin/pulsar-daemon stop broker

  2. 升级二进制文件和配置文件。

  3. 启动 broker。

cd /opt/apache-pulsar-2.9.3
./bin/pulsar-daemon start broker

#验证
jps -l
#查看端口
ss -anptl |grep 进程号
ps -ef |grep 进程号

查看集群 brokers 节点是否启动成功
./bin/pulsar-admin brokers list pulsar-cluster-prod

#输出为：
Warning: Nashorn engine is planned to be removed from a future JDK release
"10.11.0.10:8081"
"10.11.0.11:8081"
"10.11.0.12:8081"
3.1、报错

https://github.com/apache/pulsar/pull/11198

18:20:55.248 [main] ERROR org.apache.pulsar.broker.PulsarService - Failed to start Pulsar service: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /
org.apache.pulsar.metadata.api.MetadataStoreException: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /
	at org.apache.pulsar.metadata.impl.ZKMetadataStore.<init>(ZKMetadataStore.java:89) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.metadata.impl.MetadataStoreFactoryImpl.newInstance(MetadataStoreFactoryImpl.java:52) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.metadata.impl.MetadataStoreFactoryImpl.createExtended(MetadataStoreFactoryImpl.java:36) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.metadata.api.extended.MetadataStoreExtended.create(MetadataStoreExtended.java:43) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.broker.PulsarService.createLocalMetadataStore(PulsarService.java:928) ~[org.apache.pulsar-pulsar-broker-2.9.3.jar:2.9.3]
	at org.apache.pulsar.broker.PulsarService.start(PulsarService.java:621) [org.apache.pulsar-pulsar-broker-2.9.3.jar:2.9.3]
	at org.apache.pulsar.PulsarBrokerStarter$BrokerStarter.start(PulsarBrokerStarter.java:273) [org.apache.pulsar-pulsar-broker-2.9.3.jar:2.9.3]
	at org.apache.pulsar.PulsarBrokerStarter.main(PulsarBrokerStarter.java:350) [org.apache.pulsar-pulsar-broker-2.9.3.jar:2.9.3]
Caused by: org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:106) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
	at org.apache.zookeeper.ZooKeeper.addWatch(ZooKeeper.java:3192) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
	at org.apache.pulsar.metadata.impl.PulsarZooKeeperClient.access$3301(PulsarZooKeeperClient.java:74) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.metadata.impl.PulsarZooKeeperClient$23.call(PulsarZooKeeperClient.java:1130) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.metadata.impl.PulsarZooKeeperClient$23.call(PulsarZooKeeperClient.java:1124) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.metadata.impl.PulsarZooKeeperClient$ZooWorker.syncCallWithRetries(PulsarZooKeeperClient.java:1529) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.metadata.impl.PulsarZooKeeperClient.addWatch(PulsarZooKeeperClient.java:1124) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	at org.apache.pulsar.metadata.impl.ZKMetadataStore.<init>(ZKMetadataStore.java:82) ~[org.apache.pulsar-pulsar-metadata-2.9.3.jar:2.9.3]
	... 7 more
3.2、解决
2.7.1升级至2.9.3需升级zookeeper集群
参照上方步骤三.1升级zookeeper
3.3、解决完报错后再次重启升级broker
cd /opt/pulsar-2.9.3
./bin/pulsar-daemon start broker

#验证
jps -l
#查看端口
ss -anptl |grep 进程号
ps -ef |grep 进程号

#依次升级其他节点的broker
查看集群 brokers 节点是否启动成功
./bin/pulsar-admin brokers list pulsar-cluster-prod

#输出为：
Warning: Nashorn engine is planned to be removed from a future JDK release
"10.11.0.10:8081"
"10.11.0.11:8081"
"10.11.0.12:8081"
失败回滚
关闭有问题的 broker节点， 使用旧版本的二进制文件和配置文件启动 broker。

四、验证集群

通过启动consumer和producer测试消息是否能传输成功。

建议先启动Consumer，再启动Producer。

Consumer

bin/pulsar-client consume \
  persistent://public/default/test \
  -n 100 \
  -s "consumer-test" \
  -t "Exclusive"

Produce

  • 如果你指定了 pulsar-cluster-1 以外的集群名称，你需要在主题中使用不同的集群名称。
bin/pulsar-client produce \
  persistent://public/default/test \
  -n 1 \
  -m "Hello Pulsar"

结果

key:[null], properties:[], content:Hello Pulsar

要加油呀
关注
  • 1
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
RocksDB报错：Compression type Snappy is not linked with the binary.
Jermy的专栏
12-29 3505
表现：通过JNI打开RocksDB报错：snappy压缩库没有被链接： org.rocksdb.RocksDBException: Compression type Snappy is not linked with the binary. at org.rocksdb.RocksDB.open(Native Method) ~[rocksdbjni-6.6.0-fix-osx.jar:?] ...
pulsar2.8.1.zip
10-27
docker-compose安装pulsar集群本地测试环境，一键全搞定
Apache Pulsar 2.6.1 版本发布前瞻
11-15
Apache Pulsar 是 Apache 软件基金会顶级项目，是下一代云原生分布式消息流平台，集消息、存储、轻量化函数式计算为一体，采用计算与存储分离架构设计，支持多租户、持久化存储、多机房跨区域数据复制，具有强一致性、高吞吐、低延时及高可扩展性等流数据存储特性。
apache pulsar搭建过程实例，以及自己遇到的问题记录
zggpdd的博客
11-30 2732
本文目录1、Apache Pulsar 介绍2、Pulsar搭建过程2.1 pulsar集群组成2.1 准备工作安装方式1安装方式2安装方式33、部署流程3.1 zookeeper安装3.2 bookkeeper部署3.3 Broker集群部署4、遇到的问题4.1 端口被占用4.2 内存不足 1、Apache Pulsar 介绍 Pulsar 是一个用于服务器到服务器的消息系统，具有多租户、高性能等优势。 Pulsar 的关键特性如下： 1、Pulsar 的单个实例原生支持多个集群，可跨机房在集群间无缝地完
Pulsar详解2—架构和组件 & 安装和配置（示例：Go语言）
风不归的乐园
12-20 1144
Pulsar：架构和组件：分层架构、BookKeeper和ZooKeeper、Broker服务、Pulsar IO（连接器）；安装和配置：安装步骤、配置文件
Pulsar Shed Unloading后producer无法连接到broker
BigManing的博客
07-15 1558
Pulsar Shed Unloading后producer无法连接到broker 一、问题描述 业务方反馈生产消息有大量失败，我登陆查看Cat上报信息发现该Topic有5个producers，其中4个producers都有大量的失败消息，并且分布比较均匀在每个时间段都有失败，而不是集中在某个时间点。有1个producer的消息生产正常。 下图为失败上报统计信息： 二、先说原因&解决方案 ### 1、在这之前 说原因之前先普及几个概念，会让大家更易于理解这个问题的原因。如果已经了解这些概念所代表的
Apache Pulsar 在火山引擎 EMR 的集成与场景
m0_60025795的博客
05-04 138
本文介绍了火山引擎 EMR，以及我们将 Apache Pulsar 集成到火山引擎 EMR 的原因和方法，同时介绍了 Pulsar 的一些令人瞩目的优秀特性，并讨论了实时数仓和批流一体的一些典型场景、其中可能存在的痛点问题以及使用火山引擎 EMR 中的 Pulsar 结合其他组件的可能的解决方案。最后我们还展望了火山引擎 EMR 集成 Apache Pulsar 这部分工作的一些未来规划。
k8s集群部署pulsar(standalone)
weixin_45112997的博客
06-20 1228
k8s集群部署pulsar(standalone)
Centos7 安装单机模式 Apache Pulsar
极客技术宅星云-CSDN博客
06-17 1147
Centos7 安装单机模式Apache Pulsar
Kafka10：【案例】Kafka集群平滑升级
weixin_40612128的博客
03-14 969
之前我们在使用Kafka 0.9.0.0版本的时候，遇到一个比较诡异的问题 针对消费者组增加消费者的时候可能会导致rebalance，进而导致部分consumer不能再消费分区数据 意思就是之前针对这个topic的5个分区只有2个消费者消费数据，后期我动态的把消费者调整为了5个，这样可能会导致部分消费者无法消费分区中的数据。 针对这个bug这里有一份详细描述： https://issues.apache.org/jira/browse/KAFKA-2978 此bug官方在0.9.0.1版本中进行了修复 当
Pulsar集群搭建和基本命令使用
逆天骚年的博客
01-24 4398
目录1. Pulsar集群搭建2. Pulsar基本命令使用3. 使用pulsar-mamager对Pulsar进行可视化管理4. 参考资料 1. Pulsar集群搭建 为了方便快捷，此处使用Docker安装。 （1）拉取最新Pulsar Docker镜像 sudo docker pull apachepulsar/pulsar-all （2）进入Docker容器 Pulsar容器内部的8080端口和6650端口分别映射到主机的8080端口和6650端口，其中8080端口是web端口，6650用于在程序中
Apache Pulsar介绍PPT
最新发布
03-07
Apache Pulsar介绍PPT
Apache Pulsar入门demo
09-01
基于maven的用Java api编写的Producer和Consumer；Apache Pulsar入门demo，用来理解Apache Pulsar的基本原理，附带必要代码注释
Pulsar队列
11-20
Pulsar是yahoo公司的分布式高性能队列.这个是可运行程序。
【Kafka升级】-- 官方指导及注意事项
欢迎来到我的博客，一起探索代码里的世界！
06-14 1461
感谢英文原文链接： Apache Kafka如果要从2.1.x之前的版本升级，请参阅以下注释，以了解用于存储使用者偏移量的架构的更改。将inter.broker.protocol.version更改为最新版本后，将无法降级到2.1之前的版本。对于滚动升级：2.2.1中的显着变化2.2.0中的显着变化从0.8.x，0.9.x，0.10.0.x，0.10.1.x，0.10.2.x，0.11.0.x，1.0.x，1.1.x或2.0.0升级到2.1.0请注意，2.1.x包含对用于存储使用者偏移量的内部架构的更改。升
kafka 2.X版本升级到3.X
Jerelli的博客
04-25 858
kakfa版本升级
手把手教你安装Zookeeper 及可视化插件ZooInspector、ZKUI
热门推荐
战斧的博客
07-07 1万+
Zookeeper 安装 全过程，以及两个可视化插件ZooInspector、ZKUI的安装与演示
kafka单机安装
Lucifer ZHAO
08-04 2316
kafka安装准备 jdk-8u131-linux-x64.tar.gz kafka_2.13-3.2.0.tgz apache-zookeeper-3.6.1-bin.tar.gz kafka-eagle-bin-2.1.0.tar.gz zktools.zip canal.deployer-1.1.4.tar.gz
Kafka版本滚动升级（不停机）
qq_39334151的博客
02-14 3541
升级Kafka集群的版本其实很简单，核心步骤只需要4步，但是我们需要在升级的过程中确保每一步操作都不会“打扰”到producer和consumer的正常运转。注意：替换新版broker后，注意查看新版broker是否已经注册到zookeeper，所在机器上的的副本是否已经可用。注意：启动3.4.0的Kafka，由于之前配置了环境变量，注意用3.4.0目录下的程序去启动kafka,启动之后查看日志，发现在同步数据。（3）.输入命令 ls /，ls /kafka/topics可以看到topic信息。
查看pulsar的版本
07-13
要查看Pulsar的版本信息，您可以使用以下命令： ``` bin/pulsar version ``` 在Pulsar的安装目录中执行此命令，它将显示安装的Pulsar版本号。例如，输出可能类似于： ``` Pulsar version: 2.7.0 ``` 这表示您安装的是Pulsar的2.7.0版本。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值