ELK+KAFKA+XPACK+filebeat安装使用教程（史上最全）

最新推荐文章于 2024-08-27 22:55:12 发布

横飞的三角龙

最新推荐文章于 2024-08-27 22:55:12 发布

阅读量662

点赞数

文章标签：运维

本文链接：https://blog.csdn.net/dl_wdp/article/details/105995354

版权

本文档详述了一套完整的ELK (Elasticsearch, Logstash, Kibana) 集成Kafka和X-Pack的安装与配置过程，包括环境准备、组件安装、高可用架构设计。重点介绍了如何设置Elasticsearch集群、安装Elasticsearch-head插件以监控集群状态、Logstash从Kafka消费数据的配置以及Kibana的安装。还讨论了Elasticsearch的primary shard和replica的工作原理及其在查询中的角色。" 52858641,4873713,Hotspot虚拟机的垃圾收集算法与收集器解析,"['JVM', '垃圾收集器', '内存管理', 'Hotspot虚拟机']

摘要由CSDN通过智能技术生成

ELK+KAFKA+XPACK+filebeat安装使用教程（史上最全）（上）

一、架构图：

在这里插入图片描述

二、主机信息

在这里插入图片描述

三、基础环境配置

关闭防火墙、selinux，安装jdk，命令：yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel，
从网址https://elasticsearch.cn/download/，下载所需的安装包，kibana、elasticsearch、logstash、filebeat、metricbeat等。
在各主机的/etc/hosts增加域名解析配置

192.168.100.120 es1
192.168.100.121 node1
192.168.100.122 node3
192.168.100.113 node2

四、安装并配置Elasticsearch

安装elasticsearch集群服务需要奇数个，本案例采用三台服务器，安装elasticsearch（7.6.0）集群，本案例下载的tar包，解压后安装最终配置如下，本案例已破解xpack，若未破解，xpack部分可不配置，破解版除了增加认证外还有一些好用的功能如beats集中管理等，推荐破解，破解方法参考https://blog.espnlol.com/?p=504和https://blog.csdn.net/a112626290/article/details/103367867。
192.168.100.121配置

cluster.name: myelk  #集群名称
node.name: node1   #节点名称
node.master: true  
node.data: true
network.host: 192.168.100.121
http.port: 9200
discovery.seed_hosts: ["node1", "node2","node3"]
cluster.initial_master_nodes: ["node1", "node2","node3"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.ml.enabled: false
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: "Authorization,X-Requested-With,Content-Length,Content-Type"

192.168.100.113配置

cluster.name: myelk
node.name: node2
node.master: true
node.data: true
network.host: 192.168.100.113
http.port: 9200
discovery.seed_hosts: ["node1", "node2","node3"]
cluster.initial_master_nodes: ["node1", "node2","node3"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.ml.enabled: false
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: "Authorization,X-Requested-With,Content-Length,Content-