kubernetes-event-exporter+loki+grafana实现对k8s event监控管理

部署 kubernetes-event-exporter

1.Installing the Chart

#helm install kubernetes-event-exporter oci://registry-1.docker.io/bitnamicharts/kubernetes-event-exporter

2.修改配置指定kubeconfig

...
  containers:
  - args:
    - -conf=/data/config.yaml
    - -kubeconfig=/opt/k8s/.kube/config
    image: docker.io/bitnami/kubernetes-event-exporter:1.7.0-debian-12-r14
    volumeMounts:
    - mountPath: /data
      name: cfg
    - mountPath: /opt/k8s/.kube/
      name: kubeconfig
      readOnly: true
  volumes:
  - configMap:
      defaultMode: 420
      name: kubernetes-event-exporter
    name: cfg
  - name: kubeconfig
    secret:
      defaultMode: 420
      secretName: kubernetes-event-exporter-kubeconfig
...

3.修改config.yaml配置loki sink

apiVersion: v1
data:
  config.yaml: |
    leaderElection: {}
    logFormat: json
    logLevel: debug
    receivers:
    - loki:
        url: http://10.x.x.x:3100/loki/api/v1/push
        streamLabels:
          container: event-exporter
      name: loki
    route:
      routes:
      - match:
        - receiver: loki
kind: ConfigMap
...

部署loki

1.get repo

#helm repo add grafana https://grafana.github.io/helm-charts
#helm pull grafana/loki-stack --untar --version 2.6.4

2.配置

#cd loki-stack
#vim values-prod.yaml
# values-prod.yaml
loki:
  enabled: true
  replicas: 1
  rbac:
    pspEnabled: false
  persistence:
    enabled: true
    storageClassName: local-path
    size: 2Gi

promtail:
  enabled: false
  rbac:
    pspEnabled: false

3.配置pv

• 配置sc

#vim local-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-path
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer

• 制备pv

#vim pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: loki-pv
spec:
  capacity:
    storage: 2Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-path
  local:
    path: /data/loki
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: loki
          operator: In
          values:
          - enable

loki配置禁用并发请求

• kubectl get secret/loki -o yaml 将配置后解码后添加parallelise_shardable_queries: false,再编码提交

...
limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h
query_range:
  parallelise_shardable_queries: false
...

提交部署loki

#helm upgrade --install loki   -f values-prod.yaml .

curl测试

#curl -H "Content-Type: application/json"   -s -X POST "http://10.x.x.x:3100/loki/api/v1/push"   --data-raw '{"streams": [{ "stream": { "foo": "bar2" }, "values": [ [ "1726281321000000000", "fizzbuzz" ] ] }]}'