ElasticStack笔记（五）FileBeat

最新推荐文章于 2024-04-18 13:45:46 发布

L小Ray想有腮

最新推荐文章于 2024-04-18 13:45:46 发布

阅读量290

点赞数

本文链接：https://blog.csdn.net/weixin_42480750/article/details/109058742

版权

总括

在这里插入图片描述

主机	IP	HOSTNAME
FileBeat	13.13.11.5/16	web01
LogStash 1	13.13.12.5/16	logstash01
LogStash 2	13.13.12.6/16	logstash02
Elasticsearch 1	13.13.13.5/16	es01
Elasticsearch 2	13.13.13.6/16	es02
Elasticsearch 3	13.13.13.7/16	es03
Kibana	13.13.14.5/16	kibana

安装软件

参考：https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html

[root@web01 ~]# ll fil*
-rw-r--r--. 1 root root 30947458 Oct 13 20:22 filebeat-7.9.2-x86_64.rpm
-rw-r--r--. 1 root root      155 Oct 13 20:22 filebeat-7.9.2-x86_64.rpm.sha512
[root@web01 ~]# shasum -a 512 -c filebeat-7.9.2-x86_64.rpm.sha512 
filebeat-7.9.2-x86_64.rpm: OK
[root@web01 ~]# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[root@web01 ~]# rpm -iv filebeat-7.9.2-x86_64.rpm 
Verifying packages...
Preparing packages...
filebeat-7.9.2-1.x86_64
[root@web01 ~]#

Nginx日志收集

Kibana网页中可以找到各种的软件的日志收集教程

参考：http://13.13.14.5:5601/app/home#/tutorial/nginxLogs

安装配置Nginx

[root@web01 ~]# cat /etc/yum.repos.d/nginx.repo
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true
[root@web01 ~]# dnf install nginx -y
[root@web01 ~]# systemctl enable nginx
[root@web01 ~]# systemctl start nginx
[root@web01 ~]# firewall-cmd --permanent --add-service=http
[root@web01 ~]# firewall-cmd --permanent --add-service=https
[root@web01 ~]# firewall-cmd --reload

声明ES网络位置

[root@web01 ~]# tail -4 /etc/hosts
13.13.13.5 es01
13.13.13.6 es02
13.13.13.7 es03
13.13.14.5 kibana
[root@web01 ~]#

备份FileBeat配置文件

[root@web01 ~]# cp /etc/filebeat/filebeat.yml{,.bak}
[root@web01 ~]# sed -E '/#|^$/d' /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
  enabled: false
  paths:
    - /var/log/*.log
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
output.elasticsearch:
  hosts: ["localhost:9200"]
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
[root@web01 ~]# sed -E '/#|^$/d' /etc/filebeat/filebeat.yml.bak > /etc/filebeat/filebeat.yml
[root@web01 ~]#

编写FileBeat配置文件

[root@web01 ~]# vi /etc/filebeat/filebeat.yml
[root@web01 ~]# cat /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
  enabled: false
  paths:
    - /var/log/*.log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
  host: "kibana:5601"

# Configure what output to use when sending the data collected by the beat.
# ---------------------------- Elasticsearch Output -----------------------
output.elasticsearch:
  hosts: ["es01:9200", "es02:9200", "es03:9200"]

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
[root@web01 ~]#

开启Nginx模块

[root@web01 ~]# filebeat modules enable nginx
Enabled nginx
[root@web01 ~]# systemctl enable filebeat.service 
Synchronizing state of filebeat.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable filebeat
Created symlink /etc/systemd/system/multi-user.target.wants/filebeat.service → /usr/lib/systemd/system/filebeat.service.
[root@web01 ~]# systemctl start filebeat.service 
[root@web01 ~]#

模拟本地访问

[root@web01 ~]# curl localhost
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@web01 ~]#

观察日志流向

在这里插入图片描述

联合Logstash

声明Logstash位置

[root@web01 ~]# tail -6 /etc/hosts
[root@web01 ~]# tail -6 /etc/hosts
13.13.12.5 logstash01
13.13.12.6 logstash02
13.13.13.5 es01
13.13.13.6 es02
13.13.13.7 es03
13.13.14.5 kibana
[root@web01 ~]#

修改配置文件

[root@web01 ~]# cat /etc/filebeat/filebeat.yml
# ============================== Filebeat inputs =========================
filebeat.inputs:
- type: log
  # Change to true to enable this input configuration.
  enabled: true
  paths:
    - /var/log/*.log
    # Add "/var/log/messages" as target
    - /var/log/messages

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
  host: "kibana:5601"

# Configure what output to use when sending the data collected by the beat.
# ------------------------------ Logstash Output -------------------------
output.logstash:
  hosts: ["logstash01:5044", "logstash02:5044"]

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
[root@web01 ~]#

重启服务

[root@web01 ~]# filebeat modules enable logstash
Enabled logstash
[root@web01 ~]# systemctl restart filebeat.service 
[root@web01 ~]#

观察现象

在这里插入图片描述

L小Ray想有腮

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
复制链接

分享到 QQ

分享到新浪微博

扫一扫