Prometheus： ansible+consul自动发现，监控服务器

一，安装部署ansible和准备hosts文件

yum -y install ansible

注： 我把Prometheus和grafana的目录都放在新建目录/opt/monitor中，为方便管理，在其目录下创建ansible目录，所需要的文件，脚本都放在ansible下

cd /opt/monitor/
mikdir ansible

注，ansible后面的操作，都在/opt/monitor/ansible/目录下完成

vim /opt/monitor/ansible/hosts

[webservers]
192.168.123.171  name=web1

[dbservers]
192.168.123.172  name=db1
192.168.123.176  name=db2

“name=” 是后面需要用到的定义的变量，在这里放一百台服务器ip也行，看自己需求，分组

二， docker里面运行consul

单独下载consul太麻烦，我直接是拉去容器方式（容器安装步骤，在我另一篇博客的过程中 请点击查看

docker run --name consul -d -p 8500:8500 consul

注： consul可以部署在任何一台服务器上，只要能注册服务即可，我部署在Prometheus服务器上
检测是否部署成功 local hosts:8500

三， 准备ansible所需要的脚本，软件等

3.1 在ansible目录下准备node_exporter的tar包

可以去官网下载，也可以到我的百度网盘提取，百度网盘分享 提取码cjcj 请点击这里

3.2 用systemd来管理node_exporter

vim /opt/monitor/ansible/node_exporter.service

[Unit]
Description=node_exporter
[Service]
ExecStart=/usr/local/node_exporter/node_exporter
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
[Install]
WantedBy=multi-user.target

3.3 准备consul服务注册脚本

vim /opt/monitor/ansible/consul_register.sh

#!/bin/bash
service_name=$1
instance_id=$2
ip=$3
port=$4

curl -X PUT -d '{"id": "'"$instance_id"'","name": "'"$service_name"'","address": "'"$ip"'","port": "'"$port"'","tags": ["'"$service_name"'"],"checks": [{"http": "http://192.168.123.152:9100","interval": 
"5s"}]}' http://192.168.123.170:8500/v1/agent/service/register

解释：service_name 相当于Prometheus当中的job_name，相当于分组，然后脚本中tags：service_name便是consul的标签，附上图片解释一下变量

3.4 准备playbook.yml

vim /opt/monitor/ansible/playbook.yml

- hosts: webservers
  gather_facts: no
  tasks:
  - name: push node_exporter
    unarchive:
      src: node_exporter-1.4.0-rc.0.linux-amd64.tar.gz
      dest: /usr/local
  - name: rename
    shell: |
      cd /usr/local
      if [ ! -d node_exporter ]
        then mv node_exporter-1.4.0-rc.0.linux-amd64 node_exporter
      fi
  - name: copy systemd content
    copy:
      src: node_exporter.service
      dest: /usr/lib/systemd/system
  - name: start node_exporter
    systemd:
      name: node_exporter
      state: started
      enabled: yes
      daemon-reload: yes
  - name: push consul_register.sh
    copy:
      src: consul_register.sh
      dest: /usr/local/bin
  - name: register nodes into consul
    shell: /bin/bash /usr/local/bin/consul_register.sh {{ group_names[0] }} {{ name }} {{ inventory_hostname }} {{ 9100 }}
- hosts: dbservers
  gather_facts: no
  tasks:
  - name: push node_exporter
    unarchive:
      src: node_exporter-1.4.0-rc.0.linux-amd64.tar.gz
      dest: /usr/local
  - name: rename
    shell: |
      cd /usr/local
      if [ ! -d node_exporter ]
        then mv node_exporter-1.4.0-rc.0.linux-amd64 node_exporter
      fi
  - name: copy systemd content
    copy:
      src: node_exporter.service
      dest: /usr/lib/systemd/system
  - name: start node_exporter
    systemd:
      name: node_exporter
      state: started
      enabled: yes
      daemon-reload: yes
  - name: push consul_register.sh
    copy:
      src: consul_register.sh
      dest: /usr/local/bin
  - name: register nodes into consul
    shell: /bin/bash /usr/local/bin/consul_register.sh {{ group_names[0] }} {{ name }} {{ inventory_hostname }} {{ 9100 }}

四， 准备Prometheus.yml的配置文件

vim /opt/monitor/prometheus/prometheus.yml

  - job_name: 'webservers'
    consul_sd_configs:
    - server: 192.168.123.170:8500
      services: ['webservers']
  - job_name: 'dbservers'
    consul_sd_configs:
    - server: 192.168.123.170:8500
      services: ['dbservers']

注意，更改配置之后 检查Prometheus的pid，然后 kill -HUP pid 来进行reload
如果担心语法有误，可以进入到Prometheus目录下，执行:

./promtool check config prometheus.yml

五， 执行playbook.yml

执行之前做免密（博客ansible部署zabbix-agent有过详细步骤，故省略，可以点击此处），或者用ansible-playbook -i hosts -uroot -k 然后输入密码来执行。不过这种输入密码可能出现以下报错

解决思路 ： vim /etc/ansible/ansible.cfg

我用的免密，然后执行：

ansible-playbook -i hosts playbook.yml

然后去检查ip:8500 和 ip:9090 去检查即可
完结撒花