CentOS 7 搭建ELK 7.13.2 环境（二）- Beats使用

CentOS 7 搭建ELK 7.13.2 环境（二）- Beats使用

一、序言

本文主要补充介绍ELK Beats用法，ELK环境的搭建参照文章：
CentOS 7 搭建ELK 7.13.2 环境

二、metricbeat用法

1.监控elasticsearch

采集”es生产集群”的监控数据直接写入“es监控集群”

• 启用xpack.monitoring.collection.enabled

GET _cluster/settings

PUT _cluster/settings
{
  "persistent": {
    "xpack.monitoring.collection.enabled": true
  }
}

• 启用es监控模块

metricbeat modules enable elasticsearch-xpack

• 修改elasticsearch-xpack.yml

cd /etc/metricbeat
vim modules.d/elasticsearch-xpack.yml

# Module: elasticsearch
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.13/metricbeat-module-elasticsearch.html

- module: elasticsearch
  xpack.enabled: true
  period: 10s
  hosts: ["http://localhost:9200"]
  username: "elastic"
  password: "gAbEd5ViDusJgBF1p4GN"

• 禁用system模块 (根据实际情况)

metricbeat modules disable system

• 修改metricbeat.yml

cd /etc/metricbeat
vim metricbeat.yml

# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]

  # Protocol - either `http` (default) or `https`.
  #protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "gAbEd5ViDusJgBF1p4GN"

• 可视化
  
  

• 自动删除索引

通过修改metricbeat策略，实现自动删除索引

三、ILM 索引生命周期管理

1.ILM索引策略

PUT /_ilm/policy/my-ilm-policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_size": "1GB",
            "max_age": "1d",
            "max_docs": 1000
          }
        }
      },
      "warm": {
        "min_age": "30m",
        "actions": {
          "forcemerge": {
                "max_num_segments":1
              },
          "shrink": {
                "number_of_shards":1
              }
        }
      },
      "delete": {
        "min_age": "2h",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

2.索引模板

强烈建议rollover_alias别名和index_patterns保持统一，例如“metricbeat-7.13.3”对应"metricbeat-7.13.3-*"

PUT _index_template/my_index_template
{
  "index_patterns": ["metricbeat-7.13.3-*"],
  "template": {
    "settings": {
        "index.lifecycle.name": "my-ilm-policy", 
        "index.lifecycle.rollover_alias": "metricbeat-7.13.3"
    }
  },
  "priority": 0
}

3. 生命周期轮询间隔

生命周期轮询间隔 默认10分钟

PUT _cluster/settings
{
  "transient": {
    "indices.lifecycle.poll_interval":"1m"
  }
}

参考：
官网地址
冷热分离与索引生命周期管理
index-templates.html