Ansible 部署ELK集群服务

使用赛项提供的 OpenStack私有云平台，创建三台 CentOS7.9 系统的云主机分别命名为 elk-1、elk-2 和 elk-3，Ansible主机可以使用上一题的环境。要求 Ansible 节点编写剧本，执行 Ansible 剧本可以在这三个节点部署 ELK 集群服务 （在/root/目录下创建 install_elk 目录作为 ansible 工作目录，部署的入口文件命名为 install_elk.yaml) 。具体要求为三个节点均安装 Elasticsearach 服务并配置为Elasticsearch 集群； kinana安装在第一个节点； Logstash 安装在第二个节点。 （需要用到的软件包在HTTP 服务下） 完成后提交 ansible 节点的用户名、密码和 IP 地址到答题框。 （考试系统会连接到ansible 节点，执行 ansible 脚本，准备好环境，以便考试系统访问）

地址规划：

1. 基础准备

#域名解析（all node）
cat >> /etc/hosts <<EOF
10.18.4.46 ansible
10.18.4.201 elk-1
10.18.4.202 elk-2
10.18.4.203 elk-3
EOF
#####centos7.9 node#####
hostnamectl set-hostname elk-1
hostnamectl set-hostname elk-2
hostnamectl set-hostnaem elk-3
#####Ansible node#####
mkidr /root/install_elk
touch /root/install_elk/install_elk.yaml
mkdir -p
/root/install_elk/roles/{ela,kib,log}/{files,handlers,tasks,templates,vars}
#在ansible节点与三台节点建立免密钥通信
ssh-keygen
ssh-copy-id elk-1
ssh-copy-id elk-2
ssh-copy-id elk-3
#ansible主机清单
cat >> /etc/ansible/hosts <<EOF
[ela]
elk-1 node_name=elk-1
elk-2 node_name=elk-2
elk-3 node_name=elk-3
[kib]
elk-1
[log]
elk-2

2 主入口文件

cat > /root/install_elk/install_elk.yaml <<EOF
---
 1. hosts: ela
gather_facts: no
remote_user: root
roles；
 2. ela
 3. hosts: kib
gather_facts: no
remote_user: root
roles:
 4. k'i'b
 5. hosts: log
gather_facts； no
remote_user: root
roles:
 6. log
EOF

3. 部署ela集群

cat > /root/install_elk/roles/ela/tasks/main.yaml <<EOF
 1. name: 安装Java环境
unarchive:
src: jdk-8u144-linux-x64.tar.gz
dest: /opt
list_files: yes
 2. name: 添加Java环境变量
shell: echo 'export JAVA_HOST=/opt/jdk1.8.0_144' >> /etc/profile && echo
'export PATH=$PATH:$JAVA_HOME/bin' >> /etc/profile
 3. name: 生效环境变量
shell: source /etc/profile
 4. name: 创建用户
user:
name: ela
 5. name: 传输本地软件包到远程主机并且解压到指定目录
unarchive:
src: elasticsearch-7.17.0-linux-x86_64.tar.gz
dest: /opt
owner: ela
group: ela
list_files: yes
register: ret
 6. name: 创建软链接
file:
src: /opt/{{ ret.files.0 | regex_replace('/.*') }}
dest: /opt/elasticsearch
state: link
 7. name: 传输配置文件
template:
src: elasticsearch.j2
dest: /opt/elasticsearch/config/elasticsearch.yml
owner: ela
group: ela
 8. name: 传输系统配置文件
copy:
src: limits.conf
dest: /etc/security/limits.conf
 9. name: 传输系统配置文件
copy:
src: sysctl.conf
dest: /etc/sysctl.conf
 10. name: 加载 /etc/sysctl.conf文件，使内核参数生效
shell: sysctl -p
 11. name: 启动服务
# 使用ela用户执行此命令
become: yes
become_user: ela
command:
# argv 是一个列表，存放了需要执行的命令及其参数
# 一行一个
argv:
 12. nohup
 13. /opt/elasticsearch/bin/elasticsearch
 14. -d
EOF

4. elasticsearch的主配置文件模板

cat > /root/install_elk/roles/ela/templates/elasticsearch.j2 <<EOF
cluster.name: elk
node.name: {{ node_name }}
node.data: true
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts:
- 10.18.4.201
- 10.18.4.202
- 10.18.4.203
cluster.initial_master_nodes: ["elk-1","elk-2","elk-3"]
EOF

5.系统文件句柄数配置文件

cat > /root/install_elk/roles/ela/files/limits.conf <<EOF
* soft nofile 65535
* hard nofile 65535
* soft nproc 4096
* hard nproc 4096
EOF

6.系统的内核配置文件

cat > /root/install_elk/roles/ela/files/sysctl.conf <<EOF
vm.max_map_count=655360
EOF

7.部署kibana

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://10.18.4.201:9200"]
i18n.locale: "zh-CN"
pid.file: /run/kibana/kibana.pid
logging.dest: /var/log/kibana/kibana.log

8.kibana 程序进程管理模板文件

[Unit]
Description=Kibana
Documentation=https://www.elastic.co
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=ela
Group=ela
ExecStart=/opt/kibana/bin/kibana
ExecStop=/bin/pkill -F "/run/kibana/kibana.pid"
Restart=on-failure
RestartSec=3
StartLimitBurst=3
StartLimitInterval=60
WorkingDirectory=/opt/kibana
StandardOutput=journal
StandardError=inherit
[Install]
WantedBy=multi-user.target

9.部署Logstash
/root/install_elk/roles/log/tasks/main.yaml 入口文件

- name: 创建日志目录
file:
path: /var/log/logstash
state: directory
- name: 服务日志目录权限
shell: chown -R ela:ela /var/log/logstash
- name: 传输本地软件包到远程主机并且解压到指定目录
unarchive:
src: logstash-7.17.0-linux-x86_64.tar.gz
dest: /opt
list_files: yes
- name: 创建软链接
file:
src: /opt/{{ ret.files.0 | xegex_replace('/.*' }}
dest: /opt/logstash
state: link
- name: 传输配置文件
template:
src: logstash.yml
dest: /opt/logstash/config/logstash.yml
- name: 传输管道配置文件
copy:
src: logstash.conf
dest: /opt/logstash/config/logstash-sample.conf
- name: 传输系统服务文件
template:
src: logstash.service.j2
dest: /etc/systemd/system/logstash.service
- name: 启动 logstash
systemd:
name: logstash
state: started
daemon_reload: yes
- name: restart logstash
systemd:
name: logstash
state: restarted
daemon_reload: yes
#- name: 启动服务
# become: yes
# become_user: ela
# shell: sh /opt/logstash/bin/logstash -f /opt/logstash/config/logstashsample.conf
EOF

10.Logstash 程序主配置文件

cat > /root/install_elk/roles/log/templates/logstash.yml <<EOF
http.host: "0.0.0.0"
path.logs: /var/log/logstash/
EOF

11.Logstash 程序管道配置文件

cat > /root/install_elk/roles/log/files/logstash.conf <<EOF
#将本地的/var/log/yum.log内日志标准输入
input {
file {
path => "/var/log/yum.log"
type => "yum_log"
start_postion => "beginning"
}
}
#标准输出到elasticsearch中
output {
elasticsearch {
hosts => ["10.18.4.201:9200","10.18.4.202:9200","10.18.4.203:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
#user => "elastic"
#password => "changeme"
}
}
EOF

12.Logstash 程序进程管理模板

cat > /root/install_elk/roles/log/templates/logstash.service.j2 <<EOF