关闭

ES Beats安装手册

标签: logstashelasticsearch
486人阅读 评论(0) 收藏 举报
分类:

Beats安装手册


Overview

  • 该文档用于ES 5.0以上的beats系列centos/ubuntu安装手册
  • 更新时间:2017/03/18
  • Write By:Brian

Bundled Beats

Packetbeat

Function

  • 通过实时的网络包抓取分析来监控服务,主要针对服务有:
ICMP (v4 and v6)
DNS
HTTP
AMQP 0.9.1
Cassandra
Mysql
PostgreSQL
Redis
Thrift-RPC
MongoDB
Memcache

Install

  • deb
sudo apt-get install libpcap0.8
curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-5.2.2-amd64.deb
sudo dpkg -i packetbeat-5.2.2-amd64.deb
  • rpm
sudo yum install libpcap
curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-5.2.2-x86_64.rpm
sudo rpm -vi packetbeat-5.2.2-x86_64.rpm

Config

  • 配置文件位置:/etc/packetbeat/packetbeat.yml
# 配置用于捕获数据包的网络设备,any表示所有设备
#
packetbeat.interfaces.device: any
# 配置你需要监控的网络协议及端口,如果端口是标准的,则允许不配置
packetbeat.protocols.dns:
  ports: [53]
  include_authorities: true
  include_additionals: true
#
packetbeat.protocols.http:
  ports: [80, 8080, 8081, 5000, 8002]
#
packetbeat.protocols.memcache:
  ports: [11211]
#
packetbeat.protocols.mysql:
  ports: [3306]
#
packetbeat.protocols.pgsql:
  ports: [5432]
#
packetbeat.protocols.redis:
  ports: [6379]
#
packetbeat.protocols.thrift:
  ports: [9090]
#
packetbeat.protocols.mongodb:
  ports: [27017]
#
packetbeat.protocols.cassandra:
  ports: [9042]
## 配置数据端,默认为ES输出
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  hosts: ["192.168.1.42:9200"]
  username: "elastic"
  password: "BBDelastic123"
# 若需要发送至logstash,请注释ES输出配置,配置logstash路径
#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

PS:更多详细信息ES官网文档

Load Template Index

  • 默认情况下,启动会自动加载默认的index的json文件,自动创建mapping信息,如果需要配置自定义的index信息,配置如下:
output.elasticsearch:
  hosts: ["localhost:9200"]
  template.name: "packetbeat"
  template.path: "packetbeat.template.json"
  template.overwrite: false
  • 默认的,初始index已经存在,并且不会自动覆盖,如果需要覆盖配置,添加
template.overwrite: true
  • 手动上传(自定义index需要执行)
    deb or rpm:
curl -XPUT 'http://localhost:9200/_template/packetbeat' -d@/etc/packetbeat/packetbeat.template.json

Start

deb:

sudo /etc/init.d/packetbeat start

rpm:

sudo /etc/init.d/packetbeat start

Loading Sample Kibana Dashboads

  • 官网已经制作了一些现有的Dashboad,可以直接加载使用
cd /usr/share/packetbeat/
#
./scripts/import_dashboards -es http://192.168.33.60:9200
OR
./scripts/import_dashboards -es https://xyz.found.io -user user -pass password

Filebeat

Function

  • 用于收集日志目录及特定的日志文件

Install

  • deb
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.2.2-amd64.deb
sudo dpkg -i filebeat-5.2.2-amd64.deb
  • rpm
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.2.2-x86_64.rpm
sudo rpm -vi filebeat-5.2.2-x86_64.rpm

Config

  • 配置文件目录或文件(支持通配符)
vim /etc/filebeat/filebeat.yml
#
filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/*.log
    #- c:\programdata\elasticsearch\logs\*
  • 配置输出方式ES或logstash
output.elasticsearch:
  hosts: ["192.168.1.42:9200"]
  username: "elastic"
  password: "BBDelastic123"
output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

PS:需要了解更多资料点击lostash配置

Load Template Index

  • 默认情况下,启动会自动加载默认的index的json文件,自动创建mapping信息,如果需要配置自定义的index信息,配置如下:
output.elasticsearch:
  hosts: ["localhost:9200"]
  template.name: "filebeat"
  template.path: "filebeat.template.json"
  template.overwrite: false
  • 默认的,初始index已经存在,并且不会自动覆盖,如果需要覆盖配置,添加
template.overwrite: true
  • 手动上传(自定义index需要执行)
    deb or rpm:
curl -XPUT 'http://localhost:9200/_template/filebeat' -d@/etc/filebeat/filebeat.template.json

Start

  • deb
sudo /etc/init.d/filebeat start
  • rpm
sudo /etc/init.d/filebeat start

Loading The Kibana index

  • 同packetbeat
cd /usr/share/packetbeat/
#
./scripts/import_dashboards -es http://192.168.33.60:9200
OR
./scripts/import_dashboards -es https://xyz.found.io -user user -pass password

Metricbeat

Function

  • 定期的收集metrics,可以用于监控:
Apache
HAProxy
MongoDB
MySQL
Nginx
PostgreSQL
Redis
System
Zookeeper
...and so on...

Install

  • deb
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-5.2.2-amd64.deb
sudo dpkg -i metricbeat-5.2.2-amd64.deb
  • rpm
curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-5.2.2-x86_64.rpm
sudo rpm -vi metricbeat-5.2.2-x86_64.rpm

Config

  • 配置监控项及输出方式(强烈推荐查看/etc/metricbeat/metricbeat.full.yml,官网配套的完整配置文件)
metricbeat.modules:
- module: system
  metricsets:
    - cpu
    - filesystem
    - memory
    - network
    - process
  enabled: true
  period: 10s
  processes: ['.*']
  cpu_ticks: false
## apache配置样例
- module: apache
  metricsets: ["status"]
  enabled: true
  period: 1s
  hosts: ["http://127.0.0.1"]
output.elasticsearch:
  hosts: ["192.168.1.42:9200"]
  username: "elastic"
  password: "BBDelastic123"output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

Loading The Kibana index

  • 同其他Beats(默认安装不需要)(略)

Start

  • deb
sudo /etc/init.d/metricbeat start
  • rpm
sudo /etc/init.d/metricbeat start

Loading The Kibana index

  • 同packetbeat
cd /usr/share/packetbeat/
#
./scripts/import_dashboards -es http://192.168.33.60:9200
OR
./scripts/import_dashboards -es https://xyz.found.io -user user -pass password

Heartbeat

Function

  • 就像名字一样,用于检测服务的状态,但是不像是metricbeat只是检测服务器的up或者down,它可以告诉你你的服务是否可用。适用于:
ICMP
TCP
HTTP
  • Install
  • deb
curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-5.2.2-amd64.deb
sudo dpkg -i heartbeat-5.2.2-amd64.deb
  • rpm
curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-5.2.2-x86_64.rpm
sudo rpm -vi heartbeat-5.2.2-x86_64.rpm

Config

  • 配置监控项,支持配置多个分组(如:按照业务类型ping不同主机)
heartbeat.monitors:
- type: icmp
  schedule: '*/5 * * * * * *'
  hosts: ["myhost"]
- type: tcp
  schedule: '@every 5s'
  hosts: ["myhost:12345"]
  mode: any
  • 配置输出端
output.elasticsearch:
  hosts: ["192.168.1.42:9200"]
  username: "elastic"
  password: "BBDelastic123"output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

PS:更多信息查阅:heartbeat配置

Loading The Kibana index

  • 同其他Beats(默认安装不需要)(略)

Start

  • deb
sudo /etc/init.d/ start
  • rpm
sudo /etc/init.d/heartbeat start
1
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:9803次
    • 积分:193
    • 等级:
    • 排名:千里之外
    • 原创:9篇
    • 转载:1篇
    • 译文:0篇
    • 评论:4条
    文章分类
    文章存档
    最新评论