自定义prometheus-operator中容器的参数

简介

prometheus-opertator 方便了我们在Kubernetes里面部署prometheus集群，但是有一些底层的配置也被掩盖了。我遇到的问题就是配置文件的实时生效，要等3分钟才可以。于是去翻了一下CRD的定义，看到prometheus-operator提供了它使用容器的自定义参数配置。

环境

prometheus-operator  - 版本 v0.45.0

问题

创建一个Probe的监控，要等2-3分钟才可以生效，需要让配置生效快一点

kubectl apply -f probe-http.yaml

probe-http.yaml文件内容如下:

---
apiVersion: monitoring.coreos.com/v1
kind: Probe
metadata:
  name: probe-sample
  namespace: monitoring

spec:
  interval: 5s
  module: http_2xx
  jobName: probe-sample
  scrapeTimeout: 2s
  prober:
    url: blackbox-exporter:19115
    scheme: http
  targets:
    staticConfig:
      static:
      - https://www.baidu.com

但是在prometheus的配置文件中一直查看不到target.

调整容器参数

kubectl apply -f prometheus.yaml

prometheus.yaml的定义如下:

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  labels:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 2.24.0
    prometheus: k8s
  name: k8s
  namespace: monitoring
spec:
  alerting:
    alertmanagers:
    - apiVersion: v2
      name: alertmanager-main
      namespace: monitoring
      port: web
  image: quay.io/prometheus/prometheus:v2.24.0
  containers:
  - name: config-reloader  #这个地方就是自定义配置文件reloader的参数，我将默认的3分钟调整为5秒。目前支持的容器名称有prometheus,config-reloader,thanos-sidecar.
    args:
    - --watch-interval=5s
    - --listen-address=:8080
    - --reload-url=http://localhost:9090/-/reload
    - --config-file=/etc/prometheus/config/prometheus.yaml.gz
    - --config-envsubst-file=/etc/prometheus/config_out/prometheus.env.yaml
    - --watched-dir=/etc/prometheus/rules/prometheus-k8s-rulefiles-0
  nodeSelector:
    kubernetes.io/os: linux
  podMetadata:
    labels:
      app.kubernetes.io/component: prometheus
      app.kubernetes.io/name: prometheus
      app.kubernetes.io/part-of: kube-prometheus
      app.kubernetes.io/version: 2.24.0
  podMonitorNamespaceSelector: {}
  podMonitorSelector: {}
  probeNamespaceSelector: {}
  probeSelector: {}
  replicas: 1
  retention: 48h
  storage:
    volumeClaimTemplate:
      spec:  
        accessModes: 
        - ReadWriteOnce
        storageClassName: nfs-csi
        resources:
          requests:
            storage: 40Gi
      
  resources:
    requests:
      memory: 400Mi
  ruleSelector:
    matchLabels:
      prometheus: k8s
      role: alert-rules
  securityContext:
    fsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName: prometheus-k8s
  serviceMonitorNamespaceSelector: {}
  serviceMonitorSelector: {}
  version: 2.24.0

这样再创建一个新的监控目标的时候，很快就会反应到prometheus的配置文件中了。