监控 - Thanos - Prometheus的低时延与低开销数据查询
前言
假设我们必须为多个团队提供集中的指标平台。对于每个团队,我们将有一个专门的普罗米修斯。这些可能在不同环境(数据中心,区域,群集等)的同一环境中。
然后,我们将尝试提供低成本,快速的全球概览。让我们看看如何通过Thanos实现这一目标。
一、Thanos是什么?
请移步官网。
二、步骤
1.初始化Prometheus与Thanos
创建持久化存储
[root@host01 ~]# mkdir -p prometheus_data
部署Prometheus,配置文件
[root@host01 ~]# for i in `ls prometheus*`; do echo "#cat $i"; cat $i; done
#cat prometheus0.yml
global:
scrape_interval: 5s
external_labels:
cluster: eu0
replica: 0
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['127.0.0.1:9090']
#cat prometheus1.yml
global:
scrape_interval: 5s
external_labels:
cluster: eu1
replica: 0
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['127.0.0.1:9091']
#cat prometheus2.yml
global:
scrape_interval: 5s
external_labels:
cluster: eu2
replica: 0
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['127.0.0.1:9092']
运行
[root@host01 ~]# for i in $(seq 0 2); do
> docker run -d --net=host --rm \
> -v $(pwd)/prometheus"${i}".yml:/etc/prometheus/prometheus.yml \
> -v $(pwd)/prometheus_data:/prometheus"${i}" \
> -u root \
> --name prometheus"${i}" \
> quay.io/prometheus/prometheus:v2.22.2 \
> --config.file=/etc/prometheus/prometheus.yml \
> --storage.tsdb.path=/prometheus \
> --web.listen-address=:909"${i}" \
> --web.external-url=https://2886795273-909"${i}"-jago02.environments.katacoda.com \
> --web.enable-lifecycle \
> --web.enable-admin-api && echo "Prometheus ${i} started!"
> done
05a760659183552f7f068578eeb36579e1872d4aabf4f77b4c46a2f9f1525b00
Prometheus 0 started!
275561e2ee97806eba4dc194cc1849881d9b4edb4633b58f2a5fc866e337f19a
Prometheus 1 started!
d7af58f008532a758df77e65491cc6c1101a6da307886f0657acb5b84412df49
Prometheus 2 started!
[root@host01 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d7af58f00853 quay.io/prometheus/prometheus:v2.22.2 "/bin/prometheus --c…" 2 seconds ago Up 1 second prometheus2
275561e2ee97 quay.io/prometheus/prometheus:v2.22.2 "/bin/prometheus --c…" 3 seconds ago Up 3 seconds prometheus1
05a760659183 quay.io/prometheus/prometheus:v2.22.2 "/bin/prometheus --c…" 5 seconds ago Up 3 seconds prometheus0
注入Thanos Sidecars
[root@host01 ~]# for i in $(seq 0 2); do
> docker run -d --net=host --rm \
> -v $(pwd)/prometheus"${i}".yml:/etc/prometheus/prometheus.yml \
> --name prometheus-sidecar"${i}" \
> -u root \
> quay.io/thanos/thanos:v0.20.0 \
> sidecar \
> --http-address=0.0.0.0:1909"${i}" \
> --grpc-address=0.0.0.0:1919"${i}" \
> --reloader.config-file=/etc/prometheus/prometheus.yml \
> --prometheus.url=http://127.0.0.1:909"${i}" && echo "Started Thanos Sidecar for Prometheus ${i}!"
> done
a2a6c5220d4fc416ef892263c9115a28839fa8ac64c280d0478233c0fd61a857
Started Thanos Sidecar for Prometheus 0!
3dc8a335ed147054340faacdf3c44614341d992b8678c1332cae4067b74b26d6
Started Thanos Sidecar for Prometheus 1!
1c98f7b1a16864030d7e1c48d4c150e4fada5f50d7b3bb00dbd06d7a45be2d60
Started Thanos Sidecar for Prometheus 2!
[root@host01 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1c98f7b1a168 quay.io/thanos/thanos:v0.20.0 "/bin/thanos sidecar…" 3 seconds ago Up 2 seconds prometheus-sidecar2
3dc8a335ed14 quay.io/thanos/thanos:v0.20.0 "/bin/thanos sidecar…" 4 seconds ago Up 3 seconds prometheus-sidecar1
a2a6c5220d4f quay.io/thanos/thanos:v0.20.0 "/bin/thanos sidecar…" 5 seconds ago Up 4 seconds prometheus-sidecar0
d7af58f00853 quay.io/prometheus/prometheus:v2.22.2 "/bin/prometheus --c…" 11 seconds ago Up 10 seconds prometheus2
275561e2ee97 quay.io/prometheus/prometheus:v2.22.2 "/bin/prometheus --c…" 12 seconds ago Up 11 seconds prometheus1
05a760659183 quay.io/prometheus/prometheus:v2.22.2 "/bin/prometheus --c…" 14 seconds ago Up 12 seconds prometheus0
部署Querier
[root@host01 ~]# docker run -d --net=host --rm \
> --name querier \
> quay.io/thanos/thanos:v0.20.0 \
> query \
> --http-address 0.0.0.0:10912 \
> --grpc-address 0.0.0.0:10901 \
> --query.replica-label replica \
> --store 127.0.0.1:19190 \
> --store 127.0.0.1:19191 \
> --store 127.0.0.1:19192 && echo "Started Thanos Querier!"
0b936fa91739b473de179353b99466cd2ace4b4470180462c2e1f94721624d99
Started Thanos Querier!
可以访问Prometheus与Querier的web页面了。
2.使用Nginx模拟延时
配置文件
server {
listen 10902;
server_name proxy;
location / {
echo_exec @default;
}
location ^~ /api/v1/query_range {
echo_sleep 1;
echo_exec @default;
}
location @default {
proxy_pass http://127.0.0.1:10912;
}
}
运行Nginx
[root@host01 ~]# docker run -d --net=host --rm \
> -v $(pwd)/nginx.conf:/etc/nginx/conf.d/default.conf \
> --name nginx \
> yannrobert/docker-nginx && echo "Started Querier Proxy!"
201e991dbd08dc539abac84fb849579e028252ffe668f15a65a1fb4d77c1ef9c
Started Querier Proxy!
部署Thanos Query Frontend,配置文件
type: IN-MEMORY
config:
max_size: "0"
max_size_items: 2048
validity: "6h"
运行Thanos Query Frontend
[root@host01 ~]# docker run -d --net=host --rm \
> -v $(pwd)/frontend.yml:/etc/thanos/frontend.yml \
> --name query-frontend \
> quay.io/thanos/thanos:v0.20.0 \
> query-frontend \
> --http-address 0.0.0.0:20902 \
> --query-frontend.compress-responses \
> --query-frontend.downstream-url=http://127.0.0.1:10902 \
> --query-frontend.log-queries-longer-than=5s \
> --query-range.split-interval=1m \
> --query-range.response-cache-max-freshness=1m \
> --query-range.max-retries-per-request=5 \
> --query-range.response-cache-config-file=/etc/thanos/frontend.yml \
> --cache-compression-type="snappy" && echo "Started Thanos Query Frontend!"
f6e30add82bcc2a6d8b0ec0dc2845afecdf3257c090940fbba14d604ad9abf1d
Started Thanos Query Frontend!
总结
使用Thanos Query Fronted可以有效地将结果缓存在所在服务器的内存中,当后续相同请求时,数据可快速地从缓存中取出返回。