版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
<!--一个博主专栏付费入口结束-->
<link rel="stylesheet" href="https://csdnimg.cn/release/phoenix/template/css/ck_htmledit_views-d284373521.css">
<link rel="stylesheet" href="https://csdnimg.cn/release/phoenix/template/css/ck_htmledit_views-d284373521.css">
<div class="htmledit_views" id="content_views">
<h2><a name="t0"></a>一、利用Dockerfile创建Kafka和Zookeeper镜像</h2>
使用wget方式安装JDK的时候需要到Oracle官网复制临时有效路径
1.kafka.Dockerfile文件
-
FROM centos:6.6
-
-
ENV KAFKA_VERSION
"2.0.1"
-
-
RUN yum -y install vim lsof wget tar bzip2 unzip vim-enhanced passwd sudo yum-utils hostname net-tools rsync man git make automake cmake patch logrotate python-devel libpng-devel libjpeg-devel pwgen python-pip
-
-
RUN mkdir /opt/java &&\
-
wget --no-check-certificate --no-cookies --header
"Cookie: oraclelicense=accept-securebackup-cookie" https://download.oracle.com/otn-pub/java/jdk/8u201-b09/42970487e3af4f5aa5bca3f542482c60/jdk-8u201-linux-x64.tar.gz -P /opt/java
-
-
RUN mkdir /opt/kafka &&\
-
wget http://archive.apache.org/dist/kafka/
$KAFKA_VERSION/kafka_2.11-
$KAFKA_VERSION.tgz -P /opt/kafka
-
-
RUN tar zxvf /opt/java/jdk-8u201-linux-x64.tar.gz -C /opt/java &&\
-
JAVA_HOME=/opt/java/jdk1.8.0_201 &&\
-
sed -i
"/^PATH/i export JAVA_HOME=$JAVA_HOME" /root/.bash_profile &&\
-
sed -i
"s%^PATH.*$%&:$JAVA_HOME/bin%g" /root/.bash_profile &&\
-
source /root/.bash_profile
-
-
RUN tar zxvf /opt/kafka/kafka*.tgz -C /opt/kafka &&\
-
sed -i
's/num.partitions.*$/num.partitions=3/g' /opt/kafka/kafka_2.11-
$KAFKA_VERSION/config/server.properties
-
-
RUN
echo
"source /root/.bash_profile" > /opt/kafka/start.sh &&\
-
echo
"cd /opt/kafka/kafka_2.11-"
$KAFKA_VERSION >> /opt/kafka/start.sh &&\
-
echo
"sed -i 's%zookeeper.connect=.*$%zookeeper.connect=zookeeper:2181%g' /opt/kafka/kafka_2.11-"
$KAFKA_VERSION
"/config/server.properties" >> /opt/kafka/start.sh &&\
-
echo
"bin/kafka-server-start.sh config/server.properties" >> /opt/kafka/start.sh &&\
-
chmod a+x /opt/kafka/start.sh
-
-
EXPOSE 9092
-
-
ENTRYPOINT [
"sh",
"/opt/kafka/start.sh"]
2.zookeeper.Dockerfile文件
-
FROM centos:6.6
-
-
RUN yum -y install vim lsof wget tar bzip2 unzip vim-enhanced passwd sudo yum-utils hostname net-tools rsync man git make automake cmake patch logrotate python-devel libpng-devel libjpeg-devel pwgen python-pip
-
-
RUN mkdir /opt/java &&\
-
wget --no-check-certificate --no-cookies --header
"Cookie: oraclelicense=accept-securebackup-cookie" https://download.oracle.com/otn-pub/java/jdk/8u201-b09/42970487e3af4f5aa5bca3f542482c60/jdk-8u201-linux-x64.tar.gz -P /opt/java
-
-
RUN tar zxvf /opt/java/jdk-8u201-linux-x64.tar.gz -C /opt/java &&\
-
JAVA_HOME=/opt/java/jdk1.8.0_201 &&\
-
sed -i
"/^PATH/i export JAVA_HOME=$JAVA_HOME" /root/.bash_profile &&\
-
sed -i
"s%^PATH.*$%&:$JAVA_HOME/bin%g" /root/.bash_profile &&\
-
source /root/.bash_profile
-
-
ENV ZOOKEEPER_VERSION
"3.4.6"
-
-
RUN mkdir /opt/zookeeper &&\
-
wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz -P /opt/zookeeper
-
-
RUN tar zxvf /opt/zookeeper/zookeeper*.tar.gz -C /opt/zookeeper
-
-
RUN
echo
"source /root/.bash_profile" > /opt/zookeeper/start.sh &&\
-
echo
"cp /opt/zookeeper/zookeeper-"
$ZOOKEEPER_VERSION
"/conf/zoo_sample.cfg /opt/zookeeper/zookeeper-"
$ZOOKEEPER_VERSION
"/conf/zoo.cfg" >> /opt/zookeeper/start.sh &&\
-
echo
"/opt/zookeeper/zookeeper-$"ZOOKEEPER_VERSION
"/bin/zkServer.sh start-foreground" >> /opt/zookeeper/start.sh
-
-
EXPOSE 2181
-
-
ENTRYPOINT [
"sh",
"/opt/zookeeper/start.sh"]
3.执行命令
docker build -t su/zookeeper:3.4.6 -f zookeeper.Dockerfile docker build -t su/kafka:2.0.1 -f kafka.Dockerfile
二、启动镜像
-
// -itd:创建一个在后台执行的容器,同时,还能用控制台管理
-
// -h:指定容器hostname
-
// -p:映射本机的2181端口到容器的2181端口
-
docker run -itd --
name zookeeper -h zookeeper -p2181:
2181 su/zookeeper:
3.4.
6 bash
-
// --link zookeeper:将hostname为zookeeper的容器连接到kafka
-
docker run -itd --
name kafka -h kafka -p9092:
9092 --link zookeeper --env KAFKA_ZOOKEEPER_CONNECT=zookeeper:
2181 --env KAFKA_ADVERTISED_HOST_NAME=本机IP
-
--env KAFKA_ADVERTISED_PORT=
9092 su/kafka:
2.0.
1 bash
-
// Linux下查询端口号是否映射成功
-
lsof -i:
2181
-
// Windows下查询端口号是否映射成功
-
netstat -ano | findstr
"2181"
三、进行Kafka相关测试
1.进入kafka容器,创建Topic并启动消费者端
-
docker exec -it kafka bash
-
cd /opt/kafka/kafka_2
.11
-2.0
.1
-
// 环境变量
-
source /root/.bash_profile
-
// 创建Topic名为test1,指定zookeeper的hostname
-
bin/kafka-topics.sh --create --topic test1 --zookeeper zookeeper:
2181 --partitions
3 --replication-factor
1
-
// 启动消费者并监听test1
-
bin/kafka-
console-consumer.sh --zookeeper zookeeper:
2181 --topic test1
2.打开新命令行窗口,并启动生产者端,发送数据
-
docker
exec -it kafka bash
-
cd /opt/kafka/kafka_2.11-2.0.1
-
source /root/.bash_profile
-
// 启动生产者端并向名为test1的Topic发送数据
-
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test1
3.Spring Boot配置Kafka时需要配置Host文件
-
// hosts文件增加一行
-
127.0.0.1 kafka
-
// SpringBoot的yml文件中引入
-
spring:
-
kafka:
-
bootstrap-servers:
127.0.0.1:
9092