使用docker构建kafka集群demo

最新推荐文章于 2024-06-03 19:29:05 发布

是个大帅比

最新推荐文章于 2024-06-03 19:29:05 发布

阅读量214

点赞数

分类专栏： kafka 文章标签： kafka

本文链接：https://blog.csdn.net/qq_36170023/article/details/86180514

版权

kafka 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

1.从docker pull一个镜像

docker pull ubuntu:latest

2.启动ubuntu

docker run -t -i ubuntu:latest /bin/bash

3.下载jdk

# install wget
apt-get update && apt-get install -y wget
# install jdk 1.8.191
wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u191-b12/2787e4a523244c269598db4e85c51e0c/jdk-8u191-linux-x64.tar.gz 
# decompression jdk tar.gz
tar -xzvf jdk-8u191-linux-x64.tar.gz
mv jdk1.8.0_191 /usr/local/jdk

4.配置环境变量

vim /etc/environment

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:$JAVA_HOME/bin"
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export JAVA_HOME=/java/jdk1.8.0_121

修改完成之后保存关闭，并输入以下命令使环境变量立即生效

source /etc/environment

5.下载zookeeper解压并增加zoo.cfg配置文件

# install zookeeper 3.1.1
wget http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.13/zookeeper-3.4.13.tar.gz 
tar -xzvf zookeeper-3.4.13.tar.gz 
mv zookeeper-3.4.13 /usr/local/zookeeper 
mkdir -p /var/lib/zookeeper
cat > /usr/local/zookeeper/conf/zoo.cfg << EOF
    > tickTime=2000
    >dataDir=/var/lib/zookeeper
    >clientPort=2181
    >EOF

6.启动zookeeper

 /usr/local/zookeeper/bin/zkServer.sh start

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg

Starting zookeeper ... STARTED

启动成功～

用

telnet localhost 2181

连接到Zookeeper端口，发送srvr检验是否安装正确。

Trying 127.0.0.1...

Connected to localhost.

Escape character is '^]'.

srvr

Zookeeper version: 3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT

Latency min/avg/max: 0/0/0

Received: 1

Sent: 0

Connections: 1

Outstanding: 0

Zxid: 0x0

Mode: standalone

Node count: 4

Connection closed by foreign host.

tickTime=2000
dataDir=/var/lib/zookeeper
clientPort=2181
initLimit=20
syncLimit=5
server.2=172.17.0.2:2888:3888
server.4=172.17.0.4:2888:3888
server.3=172.17.0.3:2888:3888

配置 zookeeper集群报错。

This ZooKeeper instance is not currently serving requests

报这个异常的原因就是集群没有选出来leader，当集群里的结点只剩下一台，或者不足半数时，就会出现这个错误提示。

原因：网络不通，开放防火墙 2181端口选举端口3888 通信端口2888

7.下载安装 Kafka Broker

wget http://mirrors.hust.edu.cn/apache/kafka/2.1.0/kafka_2.11-1.0.2.tgz

tar -zxvf kafka_2.11-1.0.2.tgz

mv kafka_2.11-1.0.2  /usr/local/kafka

mkdir /tmp/kafka-logs

#启动kafka

/usr/local/kafka/bin/kafka-server-start.sh --daemon /usr/local/kafka/config/server.properties

8.这里启动3台机器，一定要注意myid 与配置文件的一致。开开心心今天撸kafka!