基础架构系列篇-系统centos7安装kafka

系统centos7安装kafka

1. 下载kafka

地址 http://kafka.apache.org/downloads
选择版本

点击下载
kafka_2.13-2.7.0.tgz

2. 安装与启动

以root用户登录服务器
mkdir -p /data1/kafka
使用FTP工具传kafka_2.13-2.7.0.tgz压缩包至服务器/data1/kafka目录
cd /data1/kafka
解压tar -xzvf kafka_2.13-2.7.0.tgz
cd kafka_2.13-2.7.0/
vim config/server.properties
修改参数
listeners = PLAINTEXT://localhost:9092
启动zookeeper
bin/zookeeper-server-start.sh -daemon config/zookeeper.properties &
(停止 bin/zookeeper-server-stop.sh)
查看 ps -ef | grep zookeeper

启动 kafka
bin/kafka-server-start.sh config/server.properties &
内存异常

打开vim bin/kafka-server-start.sh
修改 export KAFKA_HEAP_OPTS="-Xmx256M -Xms256M"
(默认 export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G")
常规启动 kafka
bin/kafka-server-start.sh config/server.properties &
(守护启动 nohup bin/kafka-server-start.sh config/server.properties >/dev/null 2>&1 &)
(停止 bin/kafka-server-stop.sh)
查看ps -ef | grep kafka

3. 验证

创建主题
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testkafka
参数
–create 创建
–zookeeper localhost:2181 连接zookeeper
–replication-factor 1 副本
–partitions 1 分区
–topic testkafka 主题名称
修改主题分区
bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic testkafka --partitions 2
删除主题
bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic testkafka
查看主题
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic testkafka

验证消息
打开2个终端执行
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testkafka
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testkafka --from-beginning

(不推荐)系统centos7中docker安装kafka与zookeeper(版本不兼容异常)

1. 下载

打开下载地址https://hub.docker.com
搜索kafka与zookeeper
选择镜像与tag版本
这里选wurstmeister/kafka:2.13-2.7.0
wurstmeister/zookeeper:latest
sheepkiller/kafka-manager:latest
执行
docker pull wurstmeister/kafka:2.13-2.7.0
docker pull wurstmeister/zookeeper:latest
docker pull sheepkiller/kafka-manager:latest

2. 运行

启动zookeeper
docker run -dit --name zookeeper -p 2181:2181 -v /etc/localtime:/etc/localtime wurstmeister/zookeeper

启动kafka
docker run -d --name kafka -p 9092:9092 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=宿主机ip地址:2181/kafka -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://宿主机ip地址:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -v /etc/localtime:/etc/localtime wurstmeister/kafka:2.13-2.7.0

异常

查看log

修改启动参数
-e KAFKA_HEAP_OPTS="-Xmx256m -Xms256m"

删除rq
docker rm -f kafka

启动kafka
docker run -d --name kafka -p 9092:9092 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=宿主机ip地址:2181/kafka -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://宿主机ip地址:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_HEAP_OPTS="-Xmx256m -Xms256m" -v /etc/localtime:/etc/localtime wurstmeister/kafka:2.13-2.7.0

docker run -d --name kafka -p 9092:9092 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=localhost:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_HEAP_OPTS="-Xmx256m -Xms256m" -v /etc/localtime:/etc/localtime wurstmeister/kafka:2.13-2.7.0

查看