Prometheus+Grafana+cAdvisor添加Docker容器监控

promethues：主服务
Node Exporter：收集host硬件和操作系统信息
cAdvisor：收集容器信息
grafana：仪表盘

监控端安装：
promethues+grafana+Node Exporter+cAdvisor
被监控端安装：
cAdvisor+Node Exporter

以下测试环境只用一个设备当做监控端与被监控端
192.168.120.128

步骤一、安装docker

# yum -y install docker
# vim /etc/docker/daemon.json     可配置加速器
{
  "registry-mirrors": ["xxx"]                             #填入加速器地址
}
# systemctl restart docker
# systemctl enable docker

加速器配置方法

步骤二、下载镜像包

# docker pull prom/node-exporter                       #用于机器系统数据收集
# docker pull prom/prometheus
# docker pull grafana/grafana                             #用于时序数据的可视化

步骤三、启动node-exporter

# docker run -d -p 9100:9100   -v "/proc:/host/proc:ro"   -v "/sys:/host/sys:ro"   -v "/:/rootfs:ro"   --net="host"   prom/node-exporter
# ss -anptl | grep 9100

访问url测试：http://192.168.120.128:9100/metrics

步骤四、启动prometheus

# mkdir /opt/prometheus
# cd /opt/prometheus/
# vim prometheus.yml
global:
  scrape_interval:     60s
  evaluation_interval: 60s
 
scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ['localhost:9090']
        labels:
          instance: prometheus
 
  - job_name: linux
    static_configs:
      - targets: 
        - '192.168.120.128:9100'
        labels:
          instance: localhost

  - job_name: ca
    static_configs:
      - targets: ['192.168.120.128:9400']
# docker run  -d   -p 9090:9090   -v /opt/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml    prom/prometheus          
# ss -anptl | grep 9090

访问url：http://192.168.120.128:9090/graph

步骤五、启动cAdvisor镜像

# docker run -d --publish=9400:8080 --name cadvisor -v /:/rootfs:ro -v /var/run:/var/run:rw -v /sys:/sys:ro -v /var/lib/docker/:/var/lib/docker:ro google/cadvisor:latest
# ss -anptl | grep 9400

访问url：http://192.168.120.128:9400

查看prometheus页面Targets

Centos7.4docker启动cAdvisor报错问题解决方法：

# docker logs 容器ID

报错信息如下：
Failed to start container manager: inotify_add_watch
/sys/fs/cgroup/cpuacct,cpu: no such file or directory
解决方法，执行：

# mount -o remount,rw '/sys/fs/cgroup'
# ln -s /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/cpuacct,cpu

步骤六、启动grafana

# mkdir /opt/grafana-storage
# chmod 777 -R /opt/grafana-storage
# docker run -d   -p 3000:3000   --name=grafana   -v /opt/grafana-storage:/var/lib/grafana   grafana/grafana
# ss -anptl | grep 3000

访问url：http://192.168.120.128:3000/
账号密码：admin

步骤七、添加数据源

步骤八、查找docker模板并导入
https://grafana.com/grafana/dashboards?direction=asc&orderBy=name&search=docker

此处为模板ID

导入模板
docker模板ID：11277、13112、179、11600