Beats安装手册
Overview
- 该文档用于ES 5.0以上的beats系列centos/ubuntu安装手册
更新时间
:2017/03/18Write By
:Brian
Bundled Beats
Packetbeat
Function
- 通过实时的网络包抓取分析来监控服务,主要针对服务有:
ICMP (v4 and v6)
DNS
HTTP
AMQP 0.9.1
Cassandra
Mysql
PostgreSQL
Redis
Thrift-RPC
MongoDB
Memcache
Install
- deb
sudo apt-get install libpcap0.8
curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-5.2.2-amd64.deb
sudo dpkg -i packetbeat-5.2.2-amd64.deb
- rpm
sudo yum install libpcap
curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-5.2.2-x86_64.rpm
sudo rpm -vi packetbeat-5.2.2-x86_64.rpm
Config
- 配置文件位置:
/etc/packetbeat/packetbeat.yml
# 配置用于捕获数据包的网络设备,any表示所有设备
#
packetbeat.interfaces.device: any
# 配置你需要监控的网络协议及端口,如果端口是标准的,则允许不配置
packetbeat.protocols.dns:
ports: [53]
include_authorities: true
include_additionals: true
#
packetbeat.protocols.http:
ports: [80, 8080, 8081, 5000, 8002]
#
packetbeat.protocols.memcache:
ports: [11211]
#
packetbeat.protocols.mysql:
ports: [3306]
#
packetbeat.protocols.pgsql:
ports: [5432]
#
packetbeat.protocols.redis:
ports: [6379]
#
packetbeat.protocols.thrift:
ports: [9090]
#
packetbeat.protocols.mongodb:
ports: [27017]
#
packetbeat.protocols.cassandra:
ports: [9042]
## 配置数据端,默认为ES输出
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
hosts: ["192.168.1.42:9200"]
username: "elastic"
password: "BBDelastic123"
# 若需要发送至logstash,请注释ES输出配置,配置logstash路径
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
PS
:更多详细信息ES官网文档
Load Template Index
- 默认情况下,启动会自动加载默认的index的json文件,自动创建mapping信息,如果需要配置自定义的index信息,配置如下:
output.elasticsearch:
hosts: ["localhost:9200"]
template.name: "packetbeat"
template.path: "packetbeat.template.json"
template.overwrite: false
- 默认的,初始index已经存在,并且不会自动覆盖,如果需要覆盖配置,添加
template.overwrite: true
- 手动上传(自定义index需要执行)
deb or rpm:
curl -XPUT 'http://localhost:9200/_template/packetbeat' -d@/etc/packetbeat/packetbeat.template.json
Start
deb:
sudo /etc/init.d/packetbeat start
rpm:
sudo /etc/init.d/packetbeat start
Loading Sample Kibana Dashboads
- 官网已经制作了一些现有的Dashboad,可以直接加载使用
cd /usr/share/packetbeat/
#
./scripts/import_dashboards -es http://192.168.33.60:9200
OR
./scripts/import_dashboards -es https://xyz.found.io -user user -pass password
Filebeat
Function
- 用于收集日志目录及特定的日志文件
Install
- deb
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.2.2-amd64.deb
sudo dpkg -i filebeat-5.2.2-amd64.deb
- rpm
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.2.2-x86_64.rpm
sudo rpm -vi filebeat-5.2.2-x86_64.rpm
Config
- 配置文件目录或文件(支持通配符)
vim /etc/filebeat/filebeat.yml
#
filebeat.prospectors:
- input_type: log
paths:
- /var/log/*.log
#- c:\programdata\elasticsearch\logs\*
- 配置输出方式ES或logstash
output.elasticsearch:
hosts: ["192.168.1.42:9200"]
username: "elastic"
password: "BBDelastic123"
output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
PS
:需要了解更多资料点击lostash配置
Load Template Index
- 默认情况下,启动会自动加载默认的index的json文件,自动创建mapping信息,如果需要配置自定义的index信息,配置如下:
output.elasticsearch:
hosts: ["localhost:9200"]
template.name: "filebeat"
template.path: "filebeat.template.json"
template.overwrite: false
- 默认的,初始index已经存在,并且不会自动覆盖,如果需要覆盖配置,添加
template.overwrite: true
- 手动上传(自定义index需要执行)
deb or rpm:
curl -XPUT 'http://localhost:9200/_template/filebeat' -d@/etc/filebeat/filebeat.template.json
Start
- deb
sudo /etc/init.d/filebeat start
- rpm
sudo /etc/init.d/filebeat start
Loading The Kibana index
- 同packetbeat
cd /usr/share/packetbeat/
#
./scripts/import_dashboards -es http://192.168.33.60:9200
OR
./scripts/import_dashboards -es https://xyz.found.io -user user -pass password
Metricbeat
Function
- 定期的收集metrics,可以用于监控:
Apache
HAProxy
MongoDB
MySQL
Nginx
PostgreSQL
Redis
System
Zookeeper
...and so on...
Install
- deb
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-5.2.2-amd64.deb
sudo dpkg -i metricbeat-5.2.2-amd64.deb
- rpm
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-5.2.2-x86_64.rpm
sudo rpm -vi metricbeat-5.2.2-x86_64.rpm
Config
- 配置监控项及输出方式(强烈推荐查看/etc/metricbeat/metricbeat.full.yml,官网配套的完整配置文件)
metricbeat.modules:
- module: system
metricsets:
- cpu
- filesystem
- memory
- network
- process
enabled: true
period: 10s
processes: ['.*']
cpu_ticks: false
## apache配置样例
- module: apache
metricsets: ["status"]
enabled: true
period: 1s
hosts: ["http://127.0.0.1"]
output.elasticsearch:
hosts: ["192.168.1.42:9200"]
username: "elastic"
password: "BBDelastic123"
或
output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
Loading The Kibana index
- 同其他Beats(默认安装不需要)(略)
Start
- deb
sudo /etc/init.d/metricbeat start
- rpm
sudo /etc/init.d/metricbeat start
Loading The Kibana index
- 同packetbeat
cd /usr/share/packetbeat/
#
./scripts/import_dashboards -es http://192.168.33.60:9200
OR
./scripts/import_dashboards -es https://xyz.found.io -user user -pass password
Heartbeat
Function
- 就像名字一样,用于检测服务的状态,但是不像是metricbeat只是检测服务器的up或者down,它可以告诉你你的服务是否可用。适用于:
ICMP
TCP
HTTP
- Install
- deb
curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-5.2.2-amd64.deb
sudo dpkg -i heartbeat-5.2.2-amd64.deb
- rpm
curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-5.2.2-x86_64.rpm
sudo rpm -vi heartbeat-5.2.2-x86_64.rpm
Config
- 配置监控项,支持配置多个分组(如:按照业务类型ping不同主机)
heartbeat.monitors:
- type: icmp
schedule: '*/5 * * * * * *'
hosts: ["myhost"]
- type: tcp
schedule: '@every 5s'
hosts: ["myhost:12345"]
mode: any
- 配置输出端
output.elasticsearch:
hosts: ["192.168.1.42:9200"]
username: "elastic"
password: "BBDelastic123"
或
output.logstash:
# The Logstash hosts
hosts: ["localhost:5044"]
PS
:更多信息查阅:heartbeat配置
Loading The Kibana index
- 同其他Beats(默认安装不需要)(略)
Start
- deb
sudo /etc/init.d/ start
- rpm
sudo /etc/init.d/heartbeat start