k8s 使用filebeat收集所有容器标准输出的日志

k8s-filebeat收集所有容器标准输出的日志

1. k8s-收集所有容器标准输出的日志

• filebeat-kubernetes.yaml # 采集所有容器标准输出
• app-log-stdout.yaml # 标准输出测试应用
• app-log-logfile.yaml # 日志文件测试应用

1.1 filebeat-kubernetes 配置文件

• filebeat-kubernetes采集示意图

  • 针对标准输出：以DaemonSet方式在每个Node上部署一个日志收集程序，采集/var/lib/docker/containers/目录下所有容器日志

• 示例filebeat-kubernetes.yaml配置文件

  ---
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: filebeat-config
    namespace: ops
    labels:
      k8s-app: filebeat
  data:
    filebeat.yml: |-
      filebeat.config:
        inputs:
          # Mounted `filebeat-inputs` configmap:
          path: ${path.config}/inputs.d/*.yml
          # Reload inputs configs as they change:
          reload.enabled: false
        modules:
          path: ${path.config}/modules.d/*.yml
          # Reload module configs as they change:
          reload.enabled: false
  
      output.elasticsearch:
        hosts: ['49.65.125.91:9200']
  ---
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: filebeat-inputs
    namespace: ops
    labels:
      k8s-app: filebeat
  data:
    kubernetes.yml: |-
      - type: docker
        containers.ids:
        - "*"
        processors:
          - add_kubernetes_metadata:
              in_cluster: true
  ---
  apiVersion: apps/v1 
  kind: DaemonSet
  metadata:
    name: filebeat
    namespace: ops
    labels:
      k8s-app: filebeat
  spec:
    selector:
      matchLabels:
        k8s-app: filebeat
    template:
      metadata:
        labels:
          k8s-app: filebeat
      spec:
        serviceAccountName: filebeat
        terminationGracePeriodSeconds: 30
        containers:
        - name: filebeat
          image: elastic/filebeat:7.9.2
          args: [
            "-c", "/etc/filebeat.yml",
            "-e",
          ]
          securityContext:
            runAsUser: 0
            # If using Red Hat OpenShift uncomment this:
            #privileged: true
          resources:
            limits:
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 100Mi
          volumeMounts:
          - name: config
            mountPath: /etc/filebeat.yml
            readOnly: true
            subPath: filebeat.yml
          - name: inputs
            mountPath: /usr/share/filebeat/inputs.d
            readOnly: true
          - name: data
            mountPath: