ELK日志分析系统步骤

主机名	IP 地址	操作系统	软件包
elk1	192.168.255.151	Openeuler24	Elasticsearch、logstash、kibana
elk2	192.168.255.203	Openeuler24	Elasticsearch、filebeat、httpd
elk3	192.168.255.185	Openeuler24	Elasticsearch

一、部署 ES 集群

设置 3 台 ES 主机的系统环境(以 elkl 为例，其他两台操作类似)

安装tar依赖包

dnf -y install tar

更改主机名

hostname elk1

hostname elk2

hostname elk3

在两台ES主机上设置hosts文件

三台机器同时进行以下命令

cat <<EOF>>/etc/hosts 192.168.255.151 elk1

192.168.255.203 elk2

192.168.255.185 elk3

EOF

创建es用户

三台机器同时进行以下命令

useradd es

安装java环境

dnf -y install java-11

java -version

为用户设置资源访问限制

vi /etc/security/limits.conf

es soft nofile 65535

es hard nofile 65535

es soft nproc 65535

es hard nproc 65535

es soft memlock unlimited

es hard memlock unlimited

vi /etc/sysctl.conf

vm.max_map_count=655360

sysctl -p

安装elasticsearch并编辑配置文件

tar zxf elasticsearch-7.10.0-linux-x86_64.tar.gz

ls

mv elasticsearch-7.10.0 /usr/local/elasticsearch

vi /usr/local/elasticsearch/config/jvm.options

vi /usr/local/elasticsearch/config/elasticsearch.yml

创建数据存放路径并授权

mkdir -p /elk/data

mkdir -p /elk/logs

chown -R es:es /elk/

chown -R es:es /usr/local/elasticsearch

启动es

su - es

nohup /usr/local/elasticsearch/bin/elasticsearch &

二、安装logstash

在第一台机器上安装logstash

systemctl stop firewalld

setenforce 0

yum -y install java-11

解压软件包

tar zxvf logstash-7.10.0-linux-x86_64.tar.gz

mv logstash-7.10.0 /usr/local/logstash

chmod -R 777 /usr/local/logstash/data

编译配置文件

vi /usr/local/logstash/system.conf

chmod +r /var/log/messages

#运行logstash

/usr/local/logstash/bin/logstash -f /usr/local/logstash/system.conf #在任意一台机器上采集日志

curl -XGET "http://localhost:9200/_cat/indices"

部署filebeat

dnf iy install httpd

systemctl start httpd

ls /var/log/httpd/

tar zxf filebeat-7.10.0-linux-x86_64.tar.gz

mv filebeat-7.10.0-linux-x86_64 /usr/local/filebeat

vi /data/filebeat/filebeat-7.14.0-linux-x86_64/filebeat.yml

filebeat.inputs: - type: log paths: - /var/log/httpd/access_log seccomp: enabled: false output.logstash: hosts: ["192.168.255.151:5044"]

vi /usr/local/logstash/config/beats.conf input { beats { port => "5044" codec => "json" } } output{ elasticsearch { hosts => ["192.168.255.151:9200"] index => "weblog-beat-%{+YYYY.MM.dd}" } } /usr/local/logstash/bin/logstash -f /usr/local/logstash/config/beats.conf --path.data=/usr/local/logstash/config.d/web01 &> /tmp/logstash.log &

/usr/local/filebeat/filebeat -c /usr/local/filebeat/filebeat.yml

在第一台机器部署kibana

tar zxf kibana-7.10.0-linux-x86_64.tar.gz

mv kibana7.10.0-linux-x86_64 /usr/local/kibana

vi /usr/local/kibana/config/kibana.yml

创建索引

设置索引名字

选择索引类型

更新时间，然后刷新