kafka centos/7 二进制安装

 系统环境，如果是macOS10.15这个系统建议装下面的virtualbox，否则会有一些挂载的问题

NOTE: 由于中间换了安装电脑，截图里的ip 与配置ip 不一致 ，大家安装时请注意

MacOS	10.15.7
vagrant	2.2.2
virtualbox	6.0.22

1、创建三台虚拟机，如下

2、修改三台虚拟机配置，ip，主机名，provision等有需要请自行配置

kafka-master

如需要对外提供服务，需要添加虚拟机和主机的端口映射，或开放虚拟机kafka所有节点的9092端口，这里使用端口映射的方式

Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"
  config.vm.hostname = "kafka-master"
  config.vm.network "private_network", ip: "192.168.33.40"
  config.vm.network "forwarded_port", guest: 9092, host: 9092
  config.vm.provision "shell", inline: <<-SHELL
    yum install -y wget
    wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
    yum clean all
    yum makecache
    yum install -y vim
  SHELL
end

kafka-node1

Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"
  config.vm.hostname = "kafka-node1"
  config.vm.network "private_network", ip: "192.168.33.41"
  config.vm.provision "shell", inline: <<-SHELL
    yum install -y wget
    wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
    yum clean all
    yum makecache
    yum install -y vim
  SHELL
end

kafka-node2

Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"
  config.vm.hostname = "kafka-node2"
  config.vm.network "private_network", ip: "192.168.33.42"
  config.vm.provision "shell", inline: <<-SHELL
    yum install -y wget
    wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
    yum clean all
    yum makecache
    yum install -y vim
  SHELL
end

 3、启动三台虚拟机，设置设置ssh允许root登录

cd master && vagrant up
cd node1 && vagrant up
cd node2 && vagrant up

检查ip，及主机名是否正确

 设置ssh 允许root登录

sudo vim /etc/ssh/sshd_config 

打开允许root登录 

打开密码认证

 保存退出，重启sshd服务

# 切换到root用户操作
service sshd restart

设置root 密码为123456

passwd root

 验证登录，使用filezillla，将虚拟机目录切换到/opt目录下，上传apache-zookeeper-3.5.9-bin.tar.gz，jdk-8u301-linux-x64.tar.gz，kafka_2.11-0.10.2.2.tgz安装包到三台虚拟机节点

 解压安装包

tar -zxf apache-zookeeper-3.5.9-bin.tar.gz && mv apache-zookeeper-3.5.9-bin apache-zookeeper-3.5.9

tar -zxf jdk-8u301-linux-x64.tar.gz

tar -zxf kafka_2.11-0.10.2.2.tgz

配置JAVA_HOME，KAFKA_HOME，ZOOKEEPER_HOME

vim ~/.bash_profile

# 添加以下配置
export JAVA_HOME=/opt/jdk1.8.0_301
export PATH=$PATH:$JAVA_HOME/bin
export KAFKA_HOME=/opt/kafka_2.11-0.10.2.2
export PATH=$PATH:$KAFKA_HOME/bin
export ZOOKEEPER_HOME=/opt/apache-zookeeper-3.5.9
export PATH=$PATH:$ZOOKEEPER_HOME/bin

验证java 是否安装成功，输入如下图说明安装成功

java -version

 配置hosts 添加ip 和主机名映射

vim /etc/hosts

#添加如下内容
192.168.33.40 kafka-master
192.168.33.41 kafka-node1
192.168.33.42 kafka-node2

修改zookeeper 配置文件

cd /opt/apache-zookeeper-3.5.9/conf && cp zoo_sample.cfg zoo.cfg

修改data目录如下

dataDir=/opt/apache-zookeeper-3.5.9/data 

添加服务节点地址及端口

server.0=192.168.33.40:2888:3888
server.1=192.168.33.41:2888:3888
server.2=192.168.33.42:2888:3888

修改后如下图

创建myid 文件，如下图

cd /opt/apache-zookeeper-3.5.9/data && vim myid

# 在myid文件中输入0

 

三台虚拟机myid 文件里的值不能相同，配置如下

kafka-master: 0

kafka-node1: 1

kafka-node2: 2

 三台虚拟机分别启动zk 服务

zkServer.sh start

验证是否zk 启动成功

zkServer.sh status

 安装kafka

cd /opt/kafka_2.11-0.10.2.2/config && vim server.properties

修改配置文件, broker.id 配置每个kafka节点必须不一样

broker.id=0

log.dirs=/opt/kafka_2.11-0.10.2.2/data

listeners=PLAINTEXT://192.168.33.40:9092

zookeeper.connect=localhost:2181/kafka

三个节点分别创建data目录

cd /opt/kafka_2./opt/kafka_2.11-0.10.2.2 && mkdir data

broker.id 三台节点配置如下

kafka-master: 0

kafka-node1: 1

kafka-node2: 2

listeners 三台节点配置如下

kafka-master: listeners=PLAINTEXT://192.168.33.40:9092

kafka-node1: listeners=PLAINTEXT://192.168.33.41:9092

kafka-node2: listeners=PLAINTEXT://192.168.33.42:9092

三个节点启动kafka 

 kafka-server-start.sh config/server.properties &

查看kafka 是否启动成功，登录zk查看或使用java jps命令查看

zkCli.sh

ls /kafka/brokers/ids