从零构建ELK日志分析平台:Kafka-manager安装

1. 准备

1.1 镜像准备

• zookeeper镜像

  docker pull zookeeper:3.6.1
  

• kafka-manager镜像
  此处使用我构建的kafka-manager镜像,构建步骤可参考kafka-manager-docker

  docker pull huanqingdong/kafka-manager:1.3.3.23
  

• 查看已下载镜像

  docker images
  

2. kafka-manager安装

• 2.1 创建目录/root/docker-compose/kafka-manager,用于存放kafka-manager的docker-compose.yml文件

[root@localhost ~]# mkdir -vp ~/docker-compose/kafka-manager
mkdir: 已创建目录 "/root/docker-compose/kafka-manager"

• 2.2 创建文件/root/docker-compose/kafka-manager/docker-compose.yml,内容如下

zookeeper:
  image: zookeeper:3.6.1
  container_name: kafka-manager-zk

kafka-manager:
  image: huanqingdong/kafka-manager:1.3.3.23
  container_name: kafka-manager
  ports:
    - "9000:9000"
  links:
    - zookeeper
  environment:
    ZK_HOSTS: zookeeper:2181
    KM_USERNAME: admin
    KM_PASSWORD: admin

• 2.3 启动kafka-manager

# 进入docker-compose.yml文件所在目录
[root@localhost ~]# cd /root/docker-compose/kafka-manager/
# 启动kafka-manager
[root@localhost kafka-manager]# docker-compose up -d
Creating kafka-manager-zk ... done
Creating kafka-manager ... done

# 通过docker ps查看启动情况
[root@localhost kafka]# docker ps 
CONTAINER ID        IMAGE                                 COMMAND                  CREATED             STATUS              PORTS                                                  NAMES
6eaf6c0937c4        huanqingdong/kafka-manager:1.3.3.23   "./start-kafka-man..."   2 minutes ago       Up 2 minutes        0.0.0.0:9000->9000/tcp                                 kafka-manager
b931c6b35ef9        zookeeper:3.6.1                       "/docker-entrypoin..."   2 minutes ago       Up 2 minutes        2181/tcp, 2888/tcp, 3888/tcp, 8080/tcp                 kafka-manager-zk
903022f44267        wurstmeister/kafka:2.12-2.5.0         "start-kafka.sh"         41 hours ago        Up 7 seconds        0.0.0.0:9092->9092/tcp                                 kafka
6514eb9adc20        zookeeper:3.6.1                       "/docker-entrypoin..."   41 hours ago        Up 15 minutes       2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, 8080/tcp   zookeeper

3. kafka-manager使用

• 3.1 在浏览器中输入192.168.1.14:9000,填入用户名和密码进行登入,此处用户名和密码都是admin
  
• 3.2 然后点击Cluster->Add Cluster将我们需要管理的kafka加入,
  
• 3.3 填写集群名称,集群Zookeeper地址,勾选Poll consumer information,其余均保持默认,如下图:
  
• 3.4 点击页面最下方的保存按钮,完成集群添加后,会有如下提示,然后点击Go to cluster view
  
• 3.5 此时便能看到我们的kafka集群信息了,如下所示: 集群有2个Topic,有1个Broker, 点击Topics后面的2 ,查看集群的主题信息
  
• 3.6 Topic列表中包含了我们上一章节创建的topic001,我们可以点击topic001查看他的具体信息
  
• 3.7 如下所示展示了topic001中比较详细的信息,我们重点关注最下方的Latest Offset(指示了每个分区当前消息数)和右侧的Consumers consuming from this topic(消息消费情况)

• 3.8 更多关于kafka-manager介绍,请查看github