Ansible 部署 (ELK)----Kibana/Logstash

Ansible 部署 (ELK)----Kibana/Logstash

目录结构

elk
├── deploy-kibana.yml
├── elk-pkg
│   ├── elasticsearch-7.10.0-linux-x86_64.tar.gz
│   ├── kibana-7.10.0-linux-x86_64.tar.gz
│   └── logstash-7.10.0-linux-x86_64.tar.gz
├── elk-vars.yml
└──  kibana
  ├── kibana.service.j2
  └── kibana.yml

说明:

1. Kibana 默认不允许使用 root 用户启动，因此这里需要创建普通用户: ela

2. 使用系统自带的 systemd 程序管理 Kibana 进程

3. 自定义了程序的日志路径和程序的 PID 路径

elk/deploy-kibana.yml

---
- name: 部署 Kibana
  hosts: kibana
  gather_facts: no
  remote_user: root
  vars_files:
    - elk-vars.yml
  vars:
    dirs:
      dir_log: /var/log/kibana
      dir_pid: /run/kibana

  tasks:
    - name: create user
      user:
        name: "{{ ela_user }}"
        state: present
      tags: deploy

    - name: create directory
      loop: "{{ dirs | dict2items }}"
      file:
        path: "{{ item.value }}"
        state: directory
        owner: "{{ ela_user }}"
        group: "{{ ela_user }}"
      tags: deploy

    - name: 传输本地软件包到远程主机并且解压到指定目录
      ansible.builtin.unarchive:
        src: "{{ kibana}}"
        dest: /usr/local/
        owner: "{{ ela_user }}"
        group: "{{ ela_user }}"
        list_files: yes
      register: ret
      tags:
        - deploy

    - name: 创建软链接
      ansible.builtin.file:
        src: /usr/local/{{ ret.files.0 | regex_replace('/.*') }}
        dest: /usr/local/kibana
        state: link
      tags: deploy

    - name: 传输配置文件
      template:
        src: kibana/kibana.yml
        dest: /usr/local/kibana/config/kibana.yml
      tags:
        - deploy
        - restart
      notify: restart kibana

    - name: 传输服务管理文件
      template:
        src: kibana/kibana.service.j2
        dest: /etc/systemd/system/kibana.service
      tags: deploy

    - name: 停止服务
      systemd:
        name: kibana
        state: stopped
      tags:
        - stop

    - name: 启动
      systemd:
        name: kibana
        state: started
        daemon_reload: yes
      tags:
        - deploy

  handlers:
    - name: restart kibana
      systemd:
        name: kibana
        state: restarted
      tags: restart

elk/kibana/kibana.yml

Kibana 程序的配置文件

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://es01:9200"]
pid.file: /run/kibana/kibana.pid
logging.dest: /var/log/kibana/kibana.log
i18n.locale: "zh-CN"

elk/kibana/kibana.service.j2

Kibana 程序管理模板文件

[Unit]
Description=Kibana
Documentation=https://www.elastic.co
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User={{ ela_user }}
Group={{ ela_user }}

ExecStart=/usr/local/kibana/bin/kibana

ExecStop=/bin/pkill -F "{{ dirs.dir_pid }}/kibana.pid"

Restart=on-failure
RestartSec=3

StartLimitBurst=3
StartLimitInterval=60

WorkingDirectory=/usr/local/kibana

StandardOutput=journal
StandardError=inherit

[Install]
WantedBy=multi-user.target

部署 Logstash

目录结构

elk
├── deploy-logstash.yml
│
├── elk-pkg
│   ├── elasticsearch-7.10.0-linux-x86_64.tar.gz
│   ├── kibana-7.10.0-linux-x86_64.tar.gz
│   └── logstash-7.10.0-linux-x86_64.tar.gz
├── elk-vars.yml
└── logstash
    ├── logstash.conf
    ├── logstash.service.j2
    └── logstash.yml

说明

1. 程序使用 root 用户执行
2. 指定了日志目录为 /var/log/logstash/

elk/deploy-logstash.yml

---
- name: 部署 Kibana
  hosts: logstash
  gather_facts: no
  remote_user: root
  vars_files:
    - elk-vars.yml
  tasks:
    - name: 创建日志目录
      file:
        path: /var/log/logstash
        state: directory

    - name: 传输本地软件包到远程主机并且解压到指定目录
      ansible.builtin.unarchive:
        src: "{{ logstash}}"
        dest: /usr/local/
        list_files: yes
      register: ret
      tags: deploy

    - name: 创建软链接
      ansible.builtin.file:
        src: /usr/local/{{ ret.files.0 | regex_replace('/.*') }}
        dest: /usr/local/logstash
        state: link
      tags: deploy

    - name: 传输配置文件
      template:
        src: logstash/logstash.yml
        dest: /usr/local/logstash/config/logstash.yml
      tags: deploy

    - name: 传输管道配置文件
      copy:
        src: logstash/logstash.conf
        dest: /usr/local/logstash/config/logstash-sample.conf
      tags: deploy

    - name: 传输系统服务文件
      template:
        src: logstash/logstash.service.j2
        dest: /etc/systemd/system/logstash.service
      tags: deploy

    - name: 启动 logstash
      systemd:
        name: logstash
        state: started
        daemon_reload: yes
      tags:
        - deploy

    - name: restart logstash
      systemd:
        name: logstash
        state: restarted
        daemon_reload: yes
      tags:
        - restart
...

elk/logstash/logstash.yml

Logstash 程序主配置文件

http.host: "0.0.0.0"
path.logs: /var/log/logstash/
[root@ansible elk]#

elk/logstash/logstash.conf

Logstash 程序管道配置文件

这个文件内容后面实际项目的时候，会根据情况进行修改更新

# Sample Logstash configuration for creating a simple

# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["http://es01:9200"]
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    #user => "elastic"
    #password => "changeme"
  }
}