[运维]3.containerd无法使用fluentd的问题

weixin_45396500

于 2024-10-03 13:01:30 发布

阅读量163

点赞数 1

分类专栏： k8s运维文章标签：运维

本文链接：https://blog.csdn.net/weixin_45396500/article/details/142690701

版权

k8s运维专栏收录该内容

3 篇文章 0 订阅

订阅专栏

一开始看的教程是使用dockerd的运行时配置fluentd，但是我使用的是containerd的容器运行时，这两个运行时存储日志的位置不一样，导致fluentd一开始什么日志都收集不到。

修改fluentd的deployment文件，更改Mounts挂载的位置。

containerd的挂载位置是/var/log/pods和/var/log/containers。

如下是我fluentd的挂载配置，其中还包括一个configMap的挂载，这个configMap是帮助fluentd解析containerd日志的。

volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlogcontainers
          mountPath: /var/log/containers
          readOnly: true
        - name: varlogpods
          mountPath: /var/log/pods
          readOnly: true
        - name: fluentd-config
          mountPath: /fluentd/etc/kubernetes.conf
          subPath: kubernetes.conf
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlogcontainers
        hostPath:
          path: /var/log/containers
      - name: varlogpods
        hostPath:
          path: /var/log/pods
      - name: fluentd-config
        configMap:
          name: fluentd-config
          items:
            - key: kubernetes.conf
              path: kubernetes.conf

具体的configMap如下：

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-config
  namespace: kube-logging31
data:
  kubernetes.conf: |
    <source>
      @type tail
      @id in_tail_container_logs
      path /var/log/containers/*.log
      pos_file /var/log/fluentd-containers.log.pos
      tag "#{ENV['FLUENT_CONTAINER_TAIL_TAG'] || 'kubernetes.*'}"
      exclude_path "#{ENV['FLUENT_CONTAINER_TAIL_EXCLUDE_PATH'] || ''}"
      read_from_head true
      <parse>
        @type regexp
        expression /^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<flags>[^ ]+) (?<message>.*)$/
        time_format %Y-%m-%dT%H:%M:%S.%N%:z
      </parse>
    </source>
    <source>
      @type tail
      @id in_tail_pod_logs
      path /var/log/pods/**/*.log
      pos_file /var/log/fluentd-pods.log.pos
      tag kubernetes_pods.*
      read_from_head true
      <parse>
        @type regexp
        expression /^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<flags>[^ ]+) (?<message>.*)$/
        time_format %Y-%m-%dT%H:%M:%S.%N%:z
      </parse>
    </source>
    <filter kubernetes.**>
      @type kubernetes_metadata
      kubernetes_url "https://kubernetes.default.svc"
      bearer_token_file "/var/run/secrets/kubernetes.io/serviceaccount/token"
      ca_file "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
      cache_size 1000
      cache_ttl 300
    </filter>
    <filter kubernetes.**>
      @type concat
      key log
      partial_key logtag
      partial_value P
      separator ""
    </filter>
    <match kubernetes.**>
      @type elasticsearch
      host "#{ENV['FLUENT_ELASTICSEARCH_HOST'] || 'elasticsearch-svc.kube-logging31.svc.cluster.local'}"
      port "#{ENV['FLUENT_ELASTICSEARCH_PORT'] || 9200}"
      scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
      logstash_format true
      logstash_prefix fluentd
      include_tag_key true
      type_name _doc
      logstash_dateformat %Y.%m.%d
      time_key @timestamp
      time_key_format %Y-%m-%dT%H:%M:%S.%N%:z
      reconnect_on_error true
      reload_connections true
      reload_on_failure true
      request_timeout 2147483648
    </match>

这个configmap还有问题，具体来讲识别不了kubernetes开头的数据标签。这个留到后面解决。