ZooKeeper伪集群模式安装与启动---自建用户下

---

前言

详解

---

一、关于zookeeper和kafka

二、Ubuntu下安装步骤

1.下载

https://zookeeper.apache.org/releases.html
https://kafka.apache.org/downloads

2.Ubuntu安装zookeeper

2.1步骤演示（对腾讯文档补充说明）

腾讯文档链接：https://cloud.tencent.com/developer/article/1544293

根据个人的实际文件所在目录更改/mnt/后面地址，这里将此文件解压在/usr/local/目录下

//解压
//查看，更改文件名
//创建数据文件夹
hadoop@LAPTOP-KB9G98PA:~$ sudo tar -zxf /mnt/c/tools/bigdata/apache-zookeeper-3.7.0-bin.tar.gz -C /usr/local
[sudo] password for hadoop:

hadoop@LAPTOP-KB9G98PA:~$ cd /usr/local
hadoop@LAPTOP-KB9G98PA:/usr/local$ ls
apache-zookeeper-3.7.0-bin  etc    hadoop  include  man   share
bin                         games  hbase   lib      sbin  src

hadoop@LAPTOP-KB9G98PA:/usr/local$ sudo mv ./apache-zookeeper-3.7.0-bin/ ./zookeeper
hadoop@LAPTOP-KB9G98PA:/usr/local$ ls
bin  etc  games  hadoop  hbase  include  lib  man  sbin  share  src  zookeeper

hadoop@LAPTOP-KB9G98PA:/usr/local$ cd zookeeper
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ sudo chmod -R 777 ../zookeeper
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ mkdir data

zoo_sample.cfg文件是一个示例配置代码，我们在这里复制一份并修改内容

#查看
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ ls
LICENSE.txt  NOTICE.txt  README.md  README_packaging.md  bin  conf  data  docs  lib

hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ cd conf
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ ls
configuration.xsl  log4j.properties  zoo_sample.cfg
#复制
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ cp zoo_sample.cfg zoo.cfg
#修改文件
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ vi zoo.cfg

zoo.cfg文件内容配置

tickTime = 2000
dataDir = /usr/local/zookeeper/data
clientPort = 2181
initLimit = 10
syncLimit = 5
server.1=127.0.0.1:2888:3888  
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890

#此部分应可以省略，把zookeeper加入最高权限后无需自己创建下列目录(若提示此目录不存在再自行创建)
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ mkdir zk1
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ mkdir zk2
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ mkdir zk3
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ cd zk1
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk1$ mkdir data
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk1$ cd ../zk2
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk2$ mkdir data
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk2$ cd ../zk3
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk3$ mkdir data
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk3$ cd ../conf
*************
#创建三个机器的配置文件并修改配置，主要是clientPort 和dataDir
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ cp zoo.cfg zoo1.cfg
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ cp zoo.cfg zoo2.cfg
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ cp zoo.cfg zoo3.cfg
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ vi zoo1.cfg
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ vi zoo2.cfg
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ vi zoo3.cfg
#添加myid文件，仅需输入一个数字，不能多空格或其他字符
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/conf$ cd ../zk1/data
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk1/data$ vi myid
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk1/data$ cd ..
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk1$ cd ../zk2/data
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk2/data$ vi myid
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk2/data$ cd ..
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk2$ cd ../zk3/data
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/zk3/data$ vi myid

在根端配置环境变量

hadoop@LAPTOP-KB9G98PA:~$ vim ~/.bashrc
hadoop@LAPTOP-KB9G98PA:~$ source ~/.bashrc
#添加如下
# ZOOKEEPER  路径修改
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=${ZOOKEEPER_HOME}/bin:$PATH

启动如下

hadoop@LAPTOP-KB9G98PA:~$ cd /usr/local/zookeeper
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper$ cd bin
hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/bin$ zkServer.sh start ../conf/zoo1.cfg

ZooKeeper JMX enabled by default
Using config: ../conf/zoo1.cfg
Starting zookeeper ... STARTED

2.2出现问题及解决

3.Ubuntu安装kafka

3.1安装解压配置

解压方式同zookeeper，同时将kafka加入777权限中。
修改 kafka/config目录下的server.properties

hadoop@LAPTOP-KB9G98PA:/usr/local/kafka/config$ vi server.properties

#只要检查更改
broker.id=0
log.dirs=/usr/local/kafka/logs #自己设置目录
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
num.partitions=3
#新建
default.replication.factor=3

3.2启动运行

我们学习的消息队列方式，使用了zookeeper的伪集群模式，故我们不需要使用kafka自带的zookeeper功能，提前打开zookeeper配置好的三台机器
可用jps命令检查

hadoop@LAPTOP-KB9G98PA:/usr/local/zookeeper/bin$ jps
4358 Jps
4298 QuorumPeerMain
4219 QuorumPeerMain
4158 QuorumPeerMain

cd到kafka目录下 运行

hadoop@LAPTOP-KB9G98PA:/usr/local/kafka/bin$ cd ..
hadoop@LAPTOP-KB9G98PA:/usr/local/kafka$ ./bin/kafka-server-start.sh config/server.properties &
[1] 5175
hadoop@LAPTOP-KB9G98PA:/usr/local/kafka$ jps
5619 Jps
5175 Kafka
4298 QuorumPeerMain
4219 QuorumPeerMain
4158 QuorumPeerMain

到目前为止，kafka一份已经配置完毕，在https://www.cnblogs.com/zhaoshizi/p/12154518.html
提到了复制三份并使用了软连接（存在疑问，不是很清楚三台kafka具体原理）
之后通过命令分别测试

#首先创建topic，在这里replication-factor 1被我修改为1，为3是总提示broker只启动一个，未发现原因
./bin/kafka-topics.sh --create --topic itcasttopic --partitions 3 --replication-factor 1 --zookeeper localhost:2181,localhost:2182,localhost:2183
#查看新建好的主题
./bin/kafka-topics.sh --list --zookeeper localhost:2181
#发送消息
./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic itcasttopic
#消费---*****此处输入后无反应，使用ctrl+c强制退出（有出现结果的求教）
./bin/kafka-console-consumer.sh --from-beginning --topic itcasttopic --bootstrap-server localhost:9092
#查看特定主题的详细信息
./bin/kafka-topics.sh --zookeeper localhost:2181 --describe  --topic itcasttopic
#删除
./bin/kafka-topics.sh --zookeeper localhost:2181 --delete  --topic itcasttopic

3.3问题

问题大致出在broker和localhost
留待验证

---

总结

费劲
费劲
费劲