kubernetes EFK日志管理系统

最新推荐文章于 2024-09-11 11:00:10 发布
最新推荐文章于 2024-09-11 11:00:10 发布
阅读量1.4k 收藏 1
点赞数
文章标签： 网络 大数据 json
原文链接：http://blog.51cto.com/zgui2000/2388379
版权

1. kubernetes 日志管理方案介绍:

  在kubernetes集群中一般使用EFK日志解决方案，所谓的EFK分布代表了elasticsearch、fluentd、kibana.

  fluentd主要功能是日志收集客户端，主要用途是收集每一个pod的日志，同时也会收集操作系统层面的日志，比如kubelet组件、docker服务、apiserver等组件的日志；

因此flunetd组件一般是以DaemonSet的资源类型运行在k8s之上，包括k8s master节点都需要运行fluentd容器；

  elasticsearch是日志存储功能，主要用来存储收集来的日志生成索引。一般都是要创建成ES集群。标准的ES集群一般分为Client端、Master管理端、DATA数据端。Client端主要负责对外提供接口，也就是访问ES集群就是访问Client的服务请求。然后Client再把服务转发给Master服务器，最后由Master把数据写入到DATA服务器。

  kibana主要用途是日志展示和管理搜索等。将ES集群中的日志索引通过图形界面展示出来，可以实时查看日志的内容和根据关键字搜索日志；

  有的解决方案是这样的. fluentd--->logstash(日志格式转换等)--->Redis缓存--->ES集群存储--->kibana展示。其中用到的logstash是用来做日志的过滤和格式转换的，而redis用来做缓存，主要是因为ES的写入速度很慢，但是fluentd的发送日志速度很快，如果中间没有缓存的话，可能会造成数据的延迟或者丢失；

  在k8s容器中收集日志也有集中类型，一般fluentd的日志收集客户端只能收集STDOUT(标准输出)的日志， 什么是标准输出？就是你可以通过kubectl logs -f Pod_name 查询到的日志输出就是标准输出。但是有些JAVA应用程序并没有把业务日志输出到STDOUT，而是输出到一个文件里面。
大家应该知道容器里面是不能写入文件内容的，因为容器一旦重启内容就会消失，除非通过挂载共享存储的方式才能保存文件中的日志；但是文件中的日志fluentd检测不到怎么办，这个时候就要用到sidecar容器来实现把业务日志的文件中的内容输出到标准输出；或者也可以直接修改log4j配置文件将应用日志直接输出到ES；
边角容器这种方法也有一个明显的缺陷，就是日志不仅会在原容器文件中保留下来，还会通过 stdout 输出后占用磁盘空间，这样无形中就增加了一倍磁盘空间。

2. 使用Helm安装fluentd、ES、kibana:

  由于前面几篇博文已经详细描述过helm怎么安装Redis、RabbitMQ、Mysql、Jenkins等步骤，这里就不详细描述Helm的使用方法了。
  反正大家只要记住几点就行:

  1. helm的官方网站是: https://github.com/helm/charts/tree/master/stable/ 你需要的应用都在这个目录下面；
  2. 一般的流程就是helm search 应用名---> helm fetch 应用名--->根据官方文档配置value.yaml--->启动helm应用;

  接下来我主要是把这几个应用的value.yaml文件给大家重点介绍一下：

image:
  repository: k8s.harbor.maimaiti.site/system/fluentd
## Specify an imagePullPolicy (Required)
## It's recommended to change this to 'Always' if the image tag is 'latest'
## ref: http://kubernetes.io/docs/user-guide/images/#updating-images
  tag: v2.5.1
  pullPolicy: IfNotPresent
  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  # pullSecrets:
  #   - myRegistrKeySecretName

## If using AWS Elasticsearch, all requests to ES need to be signed regardless of whether
## one is using Cognito or not. By setting this to true, this chart will install a sidecar
## proxy that takes care of signing all requests being sent to the AWS ES Domain.
awsSigningSidecar:
  enabled: false
  image:
    repository: abutaha/aws-es-proxy
    tag: 0.9

# Specify to use specific priorityClass for pods
# ref: https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
# If a Pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority
# Pods to make scheduling of the pending Pod possible.
priorityClassName: ""

## Configure resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
  # limits:
  #   cpu: 100m
  #   memory: 500Mi
  # requests:
  #   cpu: 100m
  #   memory: 200Mi

elasticsearch:
  host: 'elasticsearch-client.kube-system'
  port: 9200
  scheme: 'http'
  ssl_version: TLSv1_2
  user: ""
  password: ""
  buffer_chunk_limit: 2M
  buffer_queue_limit: 8
  logstash_prefix: 'logstash'

# If you want to add custom environment variables, use the env dict
# You can then reference these in your config file e.g.:
#     user "#{ENV['OUTPUT_USER']}"
env:
  # OUTPUT_USER: my_user
  # LIVENESS_THRESHOLD_SECONDS: 300
  # STUCK_THRESHOLD_SECONDS: 900

# If you want to add custom environment variables from secrets, use the secret list
secret:
# - name: ELASTICSEARCH_PASSWORD
#   secret_name: elasticsearch
#   secret_key: password

rbac:
  create: true

serviceAccount:
  # Specifies whether a ServiceAccount should be created
  create: true
  # The name of the ServiceAccount to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

## Specify if a Pod Security Policy for node-exporter must be created
## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/
##
podSecurityPolicy:
  enabled: false
  annotations: {}
    ## Specify pod annotations
    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#apparmor
    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#seccomp
    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#sysctl
    ##
    # seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
    # seccomp.security.alpha.kubernetes.io/defaultProfileName: 'docker/default'
    # apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default'

livenessProbe:
  enabled: true

annotations: {}

podAnnotations:
  prometheus.io/scrape: "true"
  prometheus.io/port: "24231"

## DaemonSet update strategy
## Ref: https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/
updateStrategy:
  type: RollingUpdate

tolerations:
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule

affinity: {}
  # nodeAffinity:
  #   requiredDuringSchedulingIgnoredDuringExecution:
  #     nodeSelectorTerms:
  #     - matchExpressions:
  #       - key: node-role.kubernetes.io/master
  #         operator: DoesNotExist

nodeSelector: {}

service:
  type: ClusterIP
  ports:
    - name: "monitor-agent"
      port: 24231

serviceMonitor:
  ## If true, a ServiceMonitor CRD is created for a prometheus operator
  ## https://github.com/coreos/prometheus-operator
  ##
  enabled: false
  interval: 10s
  path: /metrics
  labels: {}

prometheusRule:
  ## If true, a PrometheusRule CRD is created for a prometheus operator
  ## https://github.com/coreos/prometheus-operator
  ##
  enabled: false
  prometheusNamespace: monitoring
  labels: {}
  #  role: alert-rules

configMaps:
  system.conf: |-
    <system>
      root_dir /tmp/fluentd-buffers/
    </system>
  containers.input.conf: |-
    # This configuration file for Fluentd / td-agent is used
    # to watch changes to Docker log files. The kubelet creates symlinks that
    # capture the pod name, namespace, container name & Docker container ID
    # to the docker logs for pods in the /var/log/containers directory on the host.
    # If running this fluentd configuration in a Docker container, the /var/log
    # directory should be mounted in the container.
    #
    # These logs are then submitted to Elasticsearch which assumes the
    # installation of the fluent-plugin-elasticsearch & the
    # fluent-plugin-kubernetes_metadata_filter plugins.
    # See https://github.com/uken/fluent-plugin-elasticsearch &
    # https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter for
    # more information about the plugins.
    #
    # Example
    # =======
    # A line in the Docker log file might look like this JSON:
    #
    # {"log":"2014/09/25 21:15:03 Got request with path wombat\n",
    #  "stream":"stderr",
    #   "time":"2014-09-25T21:15:03.499185026Z"}
    #
    # The time_format specification below makes sure we properly
    # parse the time format produced by Docker. This will be
    # submitted to Elasticsearch and should appear like:
    # $ curl 'http://elasticsearch-logging:9200/_search?pretty'
    # ...
    # {
    #      "_index" : "logstash-2014.09.25",
    #      "_type" : "fluentd",
    #      "_id" : "VBrbor2QTuGpsQyTCdfzqA",
    #      "_score" : 1.0,
    #      "_source":{"log":"2014/09/25 22:45:50 Got request with path wombat\n",
    #                 "stream":"stderr","tag":"docker.container.all",
    #                 "@timestamp":"2014-09-25T22:45:50+00:00"}
    #    },
    # ...
    #
    # The Kubernetes fluentd plugin is used to write the Kubernetes metadata to the log
    # record & add labels to the log record if properly configured. This enables users
    # to filter & search logs on any metadata.
    # For example a Docker container's logs might be in the directory:
    #
    #  /var/lib/docker/containers/997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b
    #
    # and in the file:
    #
    #  997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b-json.log
    #
    # where 997599971ee6... is the Docker ID of the running container.
    # The Kubernetes kubelet makes a symbolic link to this file on the host machine
    # in the /var/log/containers directory which includes the pod name and the Kubernetes
    # container name:
    #
    #    synthetic-logger-0.25lps-pod_default_synth-lgr-997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b.log
    #    ->
    #    /var/lib/docker/containers/997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b/997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b-json.log
    #
    # The /var/log directory on the host is mapped to the /var/log directory in the container
    # running this instance of Fluentd and we end up collecting the file:
    #
    #   /var/log/containers/synthetic-logger-0.25lps-pod_default_synth-lgr-997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b.log
    #
    # This results in the tag:
    #
    #  var.log.containers.synthetic-logger-0.25lps-pod_default_synth-lgr-997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b.log
    #
    # The Kubernetes fluentd plugin is used to extract the namespace, pod name & container name
    # which are added to the log message as a kubernetes field object & the Docker container ID
    # is also added under the docker field object.
    # The final tag is:
    #
    #   kubernetes.var.log.containers.synthetic-logger-0.25lps-pod_default_synth-lgr-997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b.log
    #
    # And the final log record look like:
    #
    # {
    #   "log":"2014/09/25 21:15:03 Got request with path wombat\n",
    #   "stream":"stderr",
    #   "time":"2014-09-25T21:15:03.499185026Z",
    #   "kubernetes": {
    #     "namespace": "default",
    #     "pod_name": "synthetic-logger-0.25lps-pod",
    #     "container_name": "synth-lgr"
    #   },
    #   "docker": {
    #     "container_id": "997599971ee6366d4a5920d25b79286ad45ff37a74494f262e3bc98d909d0a7b"
    #   }
    # }
    #
    # This makes it easier for users to search for logs by pod name or by
    # the name of the Kubernetes container regardless of how many times the
    # Kubernetes pod has been restarted (resulting in a several Docker container IDs).
    # Json Log Example:
    # {"log":"[info:2016-02-16T16:04:05.930-08:00] Some log text here\n","stream":"stdout","time":"2016-02-17T00:04:05.931087621Z"}
    # CRI Log Example:
    # 2016-02-17T00:04:05.931087621Z stdout F [info:2016-02-16T16:04:05.930-08:00] Some log text here
    <source>
      @id fluentd-containers.log
      @type tail
      path /var/log/containers/*.log
      pos_file /var/log/containers.log.pos
      tag raw.kubernetes.*
      read_from_head true
      <parse>
        @type multi_format
        <pattern>
          format json
          time_key time
          time_format %Y-%m-%dT%H:%M:%S.%NZ
        </pattern>
        <pattern>
          format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
          time_format %Y-%m-%dT%H:%M:%S.%N%:z
        </pattern>
      </parse>
    </source>
    <source>
      @id nginxtest1.log
      @type tail
      path /var/log/containers/nginxtest1-*.log
      pos_file /var/log/nginxtest1.log.pos
      tag nginxtest1
      read_from_head true
      <parse>
        @type multi_format
        <pattern>
          format json
          time_key time
          time_format %Y-%m-%dT%H:%M:%S.%NZ
        </pattern>
        <pattern>
          format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
          time_format %Y-%m-%dT%H:%M:%S.%N%:z
        </pattern>
      </parse>
    </source>
    <source>
      @id httpdtest1.log
      @type tail
      path /var/log/containers/httpdtest1-*.log
      pos_file /var/log/httpdtest1.log.pos
      tag httpdtest1
      read_from_head true
      <parse>
        @type multi_format
        <pattern>
          format json
          time_key time
          time_format %Y-%m-%dT%H:%M:%S.%NZ
        </pattern>
        <pattern>
          format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
          time_format %Y-%m-%dT%H:%M:%S.%N%:z
        </pattern>
      </parse>
    </source>

    # Detect exceptions in the log output and forward them as one log entry.
    <match raw.kubernetes.**>
      @id raw.kubernetes
      @type detect_exceptions
      remove_tag_prefix raw
      message log
      stream stream
      multiline_flush_interval 5
      max_bytes 500000
      max_lines 1000
    </match>

    # Concatenate multi-line logs
    <filter **>
      @id filter_concat
      @type concat
      key message
      multiline_end_regexp /\n$/
      separator ""
    </filter>

    # Enriches records with Kubernetes metadata
    <filter kubernetes.**>
      @id filter_kubernetes_metadata
      @type kubernetes_metadata
    </filter>

    # Fixes json fields in Elasticsearch
    <filter kubernetes.**>
      @id filter_parser
      @type parser
      key_name log
      reserve_data true
      remove_key_name_field true
      <parse>
        @type multi_format
        <pattern>
          format json
        </pattern>
        <pattern>
          format none
        </pattern>
      </parse>
    </filter>

  system.input.conf: |-
    # Example:
    # 2015-12-21 23:17:22,066 [salt.state       ][INFO    ] Completed state [net.ipv4.ip_forward] at time 23:17:22.066081
    <source>
      @id minion
      @type tail
      format /^(?<time>[^ ]* [^ ,]*)[^\[]*\[[^\]]*\]\[(?<severity>[^ \]]*) *\] (?<message>.*)$/
      time_format %Y-%m-%d %H:%M:%S
      path /var/log/salt/minion
      pos_file /var/log/salt.pos
      tag salt
    </source>

    # Example:
    # Dec 21 23:17:22 gke-foo-1-1-4b5cbd14-node-4eoj startupscript: Finished running startup script /var/run/google.startup.script
    <source>
      @id startupscript.log
      @type tail
      format syslog
      path /var/log/startupscript.log
      pos_file /var/log/startupscript.log.pos
      tag startupscript
    </source>

    # Examples:
    # time="2016-02-04T06:51:03.053580605Z" level=info msg="GET /containers/json"
    # time="2016-02-04T07:53:57.505612354Z" level=error msg="HTTP Error" err="No such image: -f" statusCode=404
    # TODO(random-liu): Remove this after cri container runtime rolls out.
    <source>
      @id docker.log
      @type tail
      format /^time="(?<time>[^)]*)" level=(?<severity>[^ ]*) msg="(?<message>[^"]*)"( err="(?<error>[^"]*)")?( statusCode=($<status_code>\d+))?/
      path /var/log/docker.log
      pos_file /var/log/docker.log.pos
      tag docker
    </source>

    # Example:
    # 2016/02/04 06:52:38 filePurge: successfully removed file /var/etcd/data/member/wal/00000000000006d0-00000000010a23d1.wal
    <source>
      @id etcd.log
      @type tail
      # Not parsing this, because it doesn't have anything particularly useful to
      # parse out of it (like severities).
      format none
      path /var/log/etcd.log
      pos_file /var/log/etcd.log.pos
      tag etcd
    </source>

    # Multi-line parsing is required for all the kube logs because very large log
    # statements, such as those that include entire object bodies, get split into
    # multiple lines by glog.
    # Example:
    # I0204 07:32:30.020537    3368 server.go:1048] POST /stats/container/: (13.972191ms) 200 [[Go-http-client/1.1] 10.244.1.3:40537]
    <source>
      @id kubelet.log
      @type tail
      format multiline
      multiline_flush_interval 5s
      format_firstline /^\w\d{4}/
      format1 /^(?<severity>\w)(?<time>\d{4} [^\s]*)\s+(?<pid>\d+)\s+(?<source>[^ \]]+)\] (?<message>.*)/
      time_format %m%d %H:%M:%S.%N
      path /var/log/kubelet.log
      pos_file /var/log/kubelet.log.pos
      tag kubelet
    </source>

    # Example:
    # I1118 21:26:53.975789       6 proxier.go:1096] Port "nodePort for kube-system/default-http-backend:http" (:31429/tcp) was open before and is still needed
    <source>
      @id kube-proxy.log
      @type tail
      format multiline
      multiline_flush_interval 5s
      format_firstline /^\w\d{4}/
      format1 /^(?<severity>\w)(?<time>\d{4} [^\s]*)\s+(?<pid>\d+)\s+(?<source>[^ \]]+)\] (?<message>.*)/
      time_format %m%d %H:%M:%S.%N
      path /var/log/kube-proxy.log
      pos_file /var/log/kube-proxy.log.pos
      tag kube-proxy
    </source>

    # Example:
    # I0204 07:00:19.604280       5 handlers.go:131] GET /api/v1/nodes: (1.624207ms) 200 [[kube-controller-manager/v1.1.3 (linux/amd64) kubernetes/6a81b50] 127.0.0.1:38266]
    <source>
      @id kube-apiserver.log
      @type tail
      format multiline
      multiline_flush_interval 5s
      format_firstline /^\w\d{4}/
      format1 /^(?<severity>\w)(?<time>\d{4} [^\s]*)\s+(?<pid>\d+)\s+(?<source>[^ \]]+)\] (?<message>.*)/
      time_format %m%d %H:%M:%S.%N
      path /var/log/kube-apiserver.log
      pos_file /var/log/kube-apiserver.log.pos
      tag kube-apiserver
    </source>

    # Example:
    # I0204 06:55:31.872680       5 servicecontroller.go:277] LB already exists and doesn't need update for service kube-system/kube-ui
    <source>
      @id kube-controller-manager.log
      @type tail
      format multiline
      multiline_flush_interval 5s
      format_firstline /^\w\d{4}/
      format1 /^(?<severity>\w)(?<time>\d{4} [^\s]*)\s+(?<pid>\d+)\s+(?<source>[^ \]]+)\] (?<message>.*)/
      time_format %m%d %H:%M:%S.%N
      path /var/log/kube-controller-manager.log
      pos_file /var/log/kube-controller-manager.log.pos
      tag kube-controller-manager
    </source>

    # Example:
    # W0204 06:49:18.239674       7 reflector.go:245] pkg/scheduler/factory/factory.go:193: watch of *api.Service ended with: 401: The event in requested index is outdated and cleared (the requested history has been cleared [2578313/2577886]) [2579312]
    <source>
      @id kube-scheduler.log
      @type tail
      format multiline
      multiline_flush_interval 5s
      format_firstline /^\w\d{4}/
      format1 /^(?<severity>\w)(?<time>\d{4} [^\s]*)\s+(?<pid>\d+)\s+(?<source>[^ \]]+)\] (?<message>.*)/
      time_format %m%d %H:%M:%S.%N
      path /var/log/kube-scheduler.log
      pos_file /var/log/kube-scheduler.log.pos
      tag kube-scheduler
    </source>

    # Example:
    # I0603 15:31:05.793605       6 cluster_manager.go:230] Reading config from path /etc/gce.conf
    <source>
      @id glbc.log
      @type tail
      format multiline
      multiline_flush_interval 5s
      format_firstline /^\w\d{4}/
      format1 /^(?<severity>\w)(?<time>\d{4} [^\s]*)\s+(?<pid>\d+)\s+(?<source>[^ \]]+)\] (?<message>.*)/
      time_format %m%d %H:%M:%S.%N
      path /var/log/glbc.log
      pos_file /var/log/glbc.log.pos
      tag glbc
    </source>

    # Example:
    # TODO Add a proper example here.
    <source>
      @id cluster-autoscaler.log
      @type tail
      format multiline
      multiline_flush_interval 5s
      format_firstline /^\w\d{4}/
      format1 /^(?<severity>\w)(?<time>\d{4} [^\s]*)\s+(?<pid>\d+)\s+(?<source>[^ \]]+)\] (?<message>.*)/
      time_format %m%d %H:%M:%S.%N
      path /var/log/cluster-autoscaler.log
      pos_file /var/log/cluster-autoscaler.log.pos
      tag cluster-autoscaler
    </source>

    # Logs from systemd-journal for interesting services.
    # TODO(random-liu): Remove this after cri container runtime rolls out.
    <source>
      @id journald-docker
      @type systemd
      matches [{ "_SYSTEMD_UNIT": "docker.service" }]
      <storage>
        @type local
        persistent true
        path /var/log/journald-docker.pos
      </storage>
      read_from_head true
      tag docker
    </source>

    <source>
      @id journald-container-runtime
      @type systemd
      matches [{ "_SYSTEMD_UNIT": "{{ fluentd_container_runtime_service }}.service" }]
      <storage>
        @type local
        persistent true
        path /var/log/journald-container-runtime.pos
      </storage>
      read_from_head true
      tag container-runtime
    </source>

    <source>
      @id journald-kubelet
      @type systemd
      matches [{ "_SYSTEMD_UNIT": "kubelet.service" }]
      <storage>
        @type local
        persistent true
        path /var/log/journald-kubelet.pos
      </storage>
      read_from_head true
      tag kubelet
    </source>

    <source>
      @id journald-node-problem-detector
      @type systemd
      matches [{ "_SYSTEMD_UNIT": "node-problem-detector.service" }]
      <storage>
        @type local
        persistent true
        path /var/log/journald-node-problem-detector.pos
      </storage>
      read_from_head true
      tag node-problem-detector
    </source>

    <source>
      @id kernel
      @type systemd
      matches [{ "_TRANSPORT": "kernel" }]
      <storage>
        @type local
        persistent true
        path /var/log/kernel.pos
      </storage>
      <entry>
        fields_strip_underscores true
        fields_lowercase true
      </entry>
      read_from_head true
      tag kernel
    </source>

  forward.input.conf: |-
    # Takes the messages sent over TCP
    <source>
      @id forward
      @type forward
    </source>

  monitoring.conf: |-
    # Prometheus Exporter Plugin
    # input plugin that exports metrics
    <source>
      @id prometheus
      @type prometheus
    </source>

    <source>
      @id monitor_agent
      @type monitor_agent
    </source>

    # input plugin that collects metrics from MonitorAgent
    <source>
      @id prometheus_monitor
      @type prometheus_monitor
      <labels>
        host ${hostname}
      </labels>
    </source>

    # input plugin that collects metrics for output plugin
    <source>
      @id prometheus_output_monitor
      @type prometheus_output_monitor
      <labels>
        host ${hostname}
      </labels>
    </source>

    # input plugin that collects metrics for in_tail plugin
    <source>
      @id prometheus_tail_monitor
      @type prometheus_tail_monitor
      <labels>
        host ${hostname}
      </labels>
    </source>

  output.conf: |-
    <match nginxtest1>
      @id nginxtest1
      @type elasticsearch
      @log_level info
      include_tag_key true
      type_name _doc
      host "#{ENV['OUTPUT_HOST']}"
      port "#{ENV['OUTPUT_PORT']}"
      scheme "#{ENV['OUTPUT_SCHEME']}"
      ssl_version "#{ENV['OUTPUT_SSL_VERSION']}"
      ssl_verify true
      user "#{ENV['OUTPUT_USER']}"
      password "#{ENV['OUTPUT_PASSWORD']}"
      logstash_format true
      logstash_prefix nginxtest1
      reconnect_on_error true
      <buffer>
        @type file
        path /var/log/fluentd-buffers/nginxtest1.buffer
        flush_mode interval
        retry_type exponential_backoff
        flush_thread_count 2
        flush_interval 5s
        retry_forever
        retry_max_interval 30
        chunk_limit_size "#{ENV['OUTPUT_BUFFER_CHUNK_LIMIT']}"
        queue_limit_length "#{ENV['OUTPUT_BUFFER_QUEUE_LIMIT']}"
        overflow_action block
      </buffer>
    </match>
    <match httpdtest1>
      @id httpdtest1
      @type elasticsearch
      @log_level info
      include_tag_key true
      type_name _doc
      host "#{ENV['OUTPUT_HOST']}"
      port "#{ENV['OUTPUT_PORT']}"
      scheme "#{ENV['OUTPUT_SCHEME']}"
      ssl_version "#{ENV['OUTPUT_SSL_VERSION']}"
      ssl_verify true
      user "#{ENV['OUTPUT_USER']}"
      password "#{ENV['OUTPUT_PASSWORD']}"
      logstash_format true
      logstash_prefix httpdtest1
      reconnect_on_error true
      <buffer>
        @type file
        path /var/log/fluentd-buffers/httpdtest1.buffer
        flush_mode interval
        retry_type exponential_backoff
        flush_thread_count 2
        flush_interval 5s
        retry_forever
        retry_max_interval 30
        chunk_limit_size "#{ENV['OUTPUT_BUFFER_CHUNK_LIMIT']}"
        queue_limit_length "#{ENV['OUTPUT_BUFFER_QUEUE_LIMIT']}"
        overflow_action block
      </buffer>
    </match>
    <match **>
      @id elasticsearch
      @type elasticsearch
      @log_level info
      include_tag_key true
      type_name _doc
      host "#{ENV['OUTPUT_HOST']}"
      port "#{ENV['OUTPUT_PORT']}"
      scheme "#{ENV['OUTPUT_SCHEME']}"
      ssl_version "#{ENV['OUTPUT_SSL_VERSION']}"
      ssl_verify true
      user "#{ENV['OUTPUT_USER']}"
      password "#{ENV['OUTPUT_PASSWORD']}"
      logstash_format true
      logstash_prefix "#{ENV['LOGSTASH_PREFIX']}"
      reconnect_on_error true
      <buffer>
        @type file
        path /var/log/fluentd-buffers/kubernetes.system.buffer
        flush_mode interval
        retry_type exponential_backoff
        flush_thread_count 2
        flush_interval 5s
        retry_forever
        retry_max_interval 30
        chunk_limit_size "#{ENV['OUTPUT_BUFFER_CHUNK_LIMIT']}"
        queue_limit_length "#{ENV['OUTPUT_BUFFER_QUEUE_LIMIT']}"
        overflow_action block
      </buffer>
    </match>

# extraVolumes:
#   - name: es-certs
#     secret:
#       defaultMode: 420
#       secretName: es-certs
# extraVolumeMounts:
#   - name: es-certs
#     mountPath: /certs
#     readOnly: true

  fluentd的配置是最关键也是最复杂的，能不能日志获取成功主要看fluentd配置的是否正确;配置的重点包括如下:

  1. 按照惯例将所有的镜像改成私服镜像；
  2. 连接Es集群的配置段，重点是这里连接的是client的service；
  3. 定义了fluentd的service
  4. 重点在configMap的配置，也就是fluentd的配置文件：定义缓存目录，因为fluentd也有一个缓存文件，日志先是写入到缓存文件然后再发送给ES，如果缓存文件设置的太小容易造成堵塞；
  5. fluentd默认的配置里面已经包含了容器的日志收集/var/log/containers/*.log 因为我们的容器只要是STDOUT输出的日志，默认都会在宿主机的/var/log/containers/Pod名称开始的日志名
    只需要将每个宿主机的这个目录挂载到fluentd容器里面，fluentd容器就会采集到宿主机上面运行的所有的容器日志
  6. 如果使用fluentd的默认配置的话，所有的日志都会收集到一个索引文件，也就是默认名称为logstash-年月日的索引文件，在Kibana上面创建索引后，所有的日志都汇总在一起，如果需要
    查看单独的POD的日志，就需要自己输入查询的条件，比如按照POD名或者容器名查询。
  7. 所以我们做了一个测试，我的k8s容器里面运行了nginx应用和apahce httpd应用。然后两个应用分布设置了两个不同的索引，使用不同的TAG；这个也是我们这个EFK日志解决方案的关键之处；
  8. 除了pod的日志，fluentd的配置文件中也默认收集了kube-controller-manager、 kube-scheduler、 kube-apiserver、kube-proxy、kubelet、etcd、docker等服务的日志；
  9. output.conf文件主要针对不同的源（比如nginx和httpd两个源）做了不同的索引前缀logstash_prefix，默认的logstash_prefix就是logstash
  10. 定义输出的时候还有几个参数可能需要优化，如flush_thread_count flush_interval chunk_limit_size等关于fluentd本地缓存的参数；调整的好就不至于日志d
LAST DEPLOYED: Tue Apr 30 17:55:30 2019
NAMESPACE: kube-system
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                   DATA  AGE
fluentd-elasticsearch  6     2d

==> v1/ServiceAccount
NAME                   SECRETS  AGE
fluentd-elasticsearch  1        2d

==> v1/ClusterRole
NAME                   AGE
fluentd-elasticsearch  2d

==> v1/ClusterRoleBinding
NAME                   AGE
fluentd-elasticsearch  2d

==> v1/Service
NAME                   TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)    AGE
fluentd-elasticsearch  ClusterIP  10.200.108.50  <none>       24231/TCP  2d

==> v1/DaemonSet
NAME                   DESIRED  CURRENT  READY  UP-TO-DATE  AVAILABLE  NODE SELECTOR  AGE
fluentd-elasticsearch  8        8        8      8           8          <none>         2d

==> v1/Pod(related)
NAME                         READY  STATUS   RESTARTS  AGE
fluentd-elasticsearch-2trp8  1/1    RunningE0502 18:09:33.641998   29738 portforward.go:303] error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:39598->127.0.0.1:43154: write tcp4 127.0.0.1:39598->127.0.0.1:43154: write: broken pipe
  0         2d
fluentd-elasticsearch-2xgtb  1/1    Running  0         2d
fluentd-elasticsearch-589jc  1/1    Running  0         2d
fluentd-elasticsearch-ctkv8  1/1    Running  0         2d
fluentd-elasticsearch-d5dvz  1/1    Running  0         2d
fluentd-elasticsearch-kgdxp  1/1    Running  0         2d
fluentd-elasticsearch-r2c8h  1/1    Running  0         2d
fluentd-elasticsearch-z8p7b  1/1    Running  0         2d

NOTES:
1. To verify that Fluentd has started, run:

  kubectl --namespace=kube-system get pods -l "app.kubernetes.io/name=fluentd-elasticsearch,app.kubernetes.io/instance=fluentd-elasticsearch"

THIS APPLICATION CAPTURES ALL CONSOLE OUTPUT AND FORWARDS IT TO elasticsearch . Anything that might be identifying,
including things like IP addresses, container images, and object names will NOT be anonymized.
2. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace kube-system -l "app.kubernetes.io/name=fluentd-elasticsearch,app.kubernetes.io/instance=fluentd-elasticsearch" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl port-forward $POD_NAME 8080:80

  下面的ES集群的value.yaml配置

# Default values for elasticsearch.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
appVersion: "6.7.0"

## Define serviceAccount names for components. Defaults to component's fully qualified name.
##
serviceAccounts:
  client:
    create: true
    name:
  master:
    create: true
    name:
  data:
    create: true
    name:

## Specify if a Pod Security Policy for node-exporter must be created
## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/
##
podSecurityPolicy:
  enabled: false
  annotations: {}
    ## Specify pod annotations
    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#apparmor
    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#seccomp
    ## Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/#sysctl
    ##
    # seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
    # seccomp.security.alpha.kubernetes.io/defaultProfileName: 'docker/default'
    # apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default'

image:
  # repository: "k8s.harbor.maimaiti.site/system/elasticsearch-oss"
  repository: "k8s.harbor.maimaiti.site/system/elasticsearch"
  tag: "6.7.0"
  pullPolicy: "IfNotPresent"
  # If specified, use these secrets to access the image
  # pullSecrets:
  #   - registry-secret

testFramework:
  image: "dduportal/bats"
  tag: "0.4.0"

initImage:
  repository: "busybox"
  tag: "latest"
  pullPolicy: "Always"

cluster:
  name: "elasticsearch"
  # If you want X-Pack installed, switch to an image that includes it, enable this option and toggle the features you want
  # enabled in the environment variables outlined in the README
  xpackEnable: true
  # Some settings must be placed in a keystore, so they need to be mounted in from a secret.
  # Use this setting to specify the name of the secret
  # keystoreSecret: eskeystore
  config: {}
  # Custom parameters, as string, to be added to ES_JAVA_OPTS environment variable
  additionalJavaOpts: ""
  # Command to run at the end of deployment
  bootstrapShellCommand: ""
  env:
    # IMPORTANT: https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#minimum_master_nodes
    # To prevent data loss, it is vital to configure the discovery.zen.minimum_master_nodes setting so that each master-eligible
    # node knows the minimum number of master-eligible nodes that must be visible in order to form a cluster.
    MINIMUM_MASTER_NODES: "2"
  # List of plugins to install via dedicated init container
  plugins: []
    # - ingest-attachment
    # - mapper-size

client:
  name: client
  replicas: 2
  serviceType: ClusterIP
  ## If coupled with serviceType = "NodePort", this will set a specific nodePort to the client HTTP port
  # httpNodePort: 30920
  loadBalancerIP: {}
  loadBalancerSourceRanges: {}
## (dict) If specified, apply these annotations to the client service
#  serviceAnnotations:
#    example: client-svc-foo
  heapSize: "512m"
  # additionalJavaOpts: "-XX:MaxRAM=512m"
  antiAffinity: "soft"
  nodeAffinity: {}
  nodeSelector: {}
  tolerations: []
  initResources: {}
    # limits:
    #   cpu: "25m"
    #   # memory: "128Mi"
    # requests:
    #   cpu: "25m"
    #   memory: "128Mi"
  resources:
    limits:
      cpu: "1"
      # memory: "1024Mi"
    requests:
      cpu: "25m"
      memory: "512Mi"
  priorityClassName: ""
  ## (dict) If specified, apply these annotations to each client Pod
  # podAnnotations:
  #   example: client-foo
  podDisruptionBudget:
    enabled: false
    minAvailable: 1
    # maxUnavailable: 1
  ingress:
    enabled: false
    # user: NAME
    # password: PASSWORD
    annotations: {}
      # kubernetes.io/ingress.class: nginx
      # kubernetes.io/tls-acme: "true"
    path: /
    hosts:
      - chart-example.local
    tls: []
    #  - secretName: chart-example-tls
    #    hosts:
    #      - chart-example.local

master:
  name: master
  exposeHttp: false
  replicas: 3
  heapSize: "512m"
  # additionalJavaOpts: "-XX:MaxRAM=512m"
  persistence:
    enabled: false
    accessMode: ReadWriteOnce
    name: data
    size: "4Gi"
    storageClass: "dynamic"
  readinessProbe:
    httpGet:
      path: /_cluster/health?local=true
      port: 9200
    initialDelaySeconds: 5
  antiAffinity: "soft"
  nodeAffinity: {}
  nodeSelector: {}
  tolerations: []
  initResources: {}
    # limits:
    #   cpu: "25m"
    #   # memory: "128Mi"
    # requests:
    #   cpu: "25m"
    #   memory: "128Mi"
  resources:
    limits:
      cpu: "1"
      # memory: "1024Mi"
    requests:
      cpu: "25m"
      memory: "512Mi"
  priorityClassName: ""
  ## (dict) If specified, apply these annotations to each master Pod
  # podAnnotations:
  #   example: master-foo
  podManagementPolicy: OrderedReady
  podDisruptionBudget:
    enabled: false
    minAvailable: 2  # Same as `cluster.env.MINIMUM_MASTER_NODES`
    # maxUnavailable: 1
  updateStrategy:
    type: OnDelete

data:
  name: data
  exposeHttp: false
  replicas: 2
  heapSize: "1536m"
  # additionalJavaOpts: "-XX:MaxRAM=1536m"
  persistence:
    enabled: false
    accessMode: ReadWriteOnce
    name: data
    size: "30Gi"
    storageClass: "dynamic"
  readinessProbe:
    httpGet:
      path: /_cluster/health?local=true
      port: 9200
    initialDelaySeconds: 5
  terminationGracePeriodSeconds: 3600
  antiAffinity: "soft"
  nodeAffinity: {}
  nodeSelector: {}
  tolerations: []
  initResources: {}
    # limits:
    #   cpu: "25m"
    #   # memory: "128Mi"
    # requests:
    #   cpu: "25m"
    #   memory: "128Mi"
  resources:
    limits:
      cpu: "1"
      # memory: "2048Mi"
    requests:
      cpu: "25m"
      memory: "1536Mi"
  priorityClassName: ""
  ## (dict) If specified, apply these annotations to each data Pod
  # podAnnotations:
  #   example: data-foo
  podDisruptionBudget:
    enabled: false
    # minAvailable: 1
    maxUnavailable: 1
  podManagementPolicy: OrderedReady
  updateStrategy:
    type: OnDelete
  hooks:  # post-start and pre-stop hooks
    drain:  # drain the node before stopping it and re-integrate it into the cluster after start
      enabled: true

## Sysctl init container to setup vm.max_map_count
# see https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html
# and https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration-memory.html#mlockall
sysctlInitContainer:
  enabled: true
## Additional init containers
extraInitContainers: |

  关于ES集群的配置，主要包含如下:

  1. 经镜像都改为私服镜像；
  2. 定义ES集群的名字；
  3. 定义client服务Pod的名字，副本数，jvm内存使用量;
  4. 定义master服务Pod的名字,副本数，master一般为奇数，3或者5，最好开启持久化存储，使用ceph RBD的sc
  5. 定义data节点的POD的名字，副本数，最好开启持久化存储；注意数据节点这里开启了hooks。就是数据节点
    的停止和启动，需要有顺序，并且在脱离集群之前需要做一些配置；
    6、 kibana和fluented连接的都是elasticsearch-client的svc;9200是提供服务的端口，9300是集群端口。还有一个
    elasticsearch-discovery的无头服务的用途就是每个POD的名称需要固定下来，及时重启了ES节点的名字也不会变化。 
[root@master-01 fluentd-elasticsearch]# helm status elasticsearch
LAST DEPLOYED: Tue Apr 30 17:17:13 2019
NAMESPACE: kube-system
STATUS: DEPLOYED

RESOURCES:
==> v1/ConfigMap
NAME                DATA  AGE
elasticsearch       4     2d
elasticsearch-test  1     2d

==> v1/ServiceAccount
NAME                  SECRETS  AGE
elasticsearch-client  1        2d
elasticsearch-data    1        2d
elasticsearch-master  1        2d

==> v1/Service
NAME                     TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)   AGE
elasticsearch-client     ClusterIP  10.200.180.10  <none>       9200/TCP  2d
elasticsearch-discovery  ClusterIP  None           <none>       9300/TCP  2d

==> v1beta1/Deployment
NAME                  DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
elasticsearch-client  2        2        2           2          2d

==> v1beta1/StatefulSet
NAME                  DESIRED  CURRENT  AGE
elasticsearch-data    2        2        2d
elasticsearch-master  3        3        2d

==> v1/Pod(related)
NAME                                   READY  STATUS   RESTARTS  AGE
elasticsearch-client-6bb89766f9-wfbxh  1/1    Running  0         2d
elasticsearch-client-6bb89766f9-xvz6c  1/1    Running  0         2d
elasticsearch-data-0                   1/1    Running  0         2d
elasticsearch-data-1                   1/1    Running  0         2d
elasticsearch-master-0                 1/1    Running  0         2d
elasticsearch-master-1                 1/1    Running  0         2d
elasticsearch-master-2                 1/1    Running  0         2d

NOTES:
The elasticsearch cluster has been installed.

Elasticsearch can be accessed:

  * Within your cluster, at the following DNS name at port 9200:

    elasticsearch-client.kube-system.svc

  * From outside the cluster, run these commands in the same shell:

    export POD_NAME=$(kubectl get pods --namespace kube-system -l "app=elasticsearch,component=client,release=elasticsearch" -o jsonpath="{.items[0].metadata.name}")
    echo "Visit http://127.0.0.1:9200 to use Elasticsearch"
    kubectl port-forward --namespace kube-system $POD_NAME 9200:9200

image:
  # repository: "k8s.harbor.maimaiti.site/system/kibana-oss"
  repository: "k8s.harbor.maimaiti.site/system/kibana"
  tag: "6.7.0"
  pullPolicy: "IfNotPresent"

testFramework:
  image: "dduportal/bats"
  tag: "0.4.0"

commandline:
  args: []

env: {}
  # All Kibana configuration options are adjustable via env vars.
  # To adjust a config option to an env var uppercase + replace `.` with `_`
  # Ref: https://www.elastic.co/guide/en/kibana/current/settings.html
  #
  # ELASTICSEARCH_URL: http://elasticsearch-client:9200
  # SERVER_PORT: 5601
  # LOGGING_VERBOSE: "true"
  # SERVER_DEFAULTROUTE: "/app/kibana"

files:
  kibana.yml:
    ## Default Kibana configuration from kibana-docker.
    server.name: kibana
    server.host: "0"
    elasticsearch.url: http://elasticsearch-client.kube-system:9200

    ## Custom config properties below
    ## Ref: https://www.elastic.co/guide/en/kibana/current/settings.html
    # server.port: 5601
    # logging.verbose: "true"
    # server.defaultRoute: "/app/kibana"

deployment:
  annotations: {}

service:
  type: NodePort
  nodePort: 30001
  # clusterIP: None
  # portName: kibana-svc
  externalPort: 443
  internalPort: 5601
  # authProxyPort: 5602 To be used with authProxyEnabled and a proxy extraContainer
  ## External IP addresses of service
  ## Default: nil
  ##
  # externalIPs:
  # - 192.168.0.1
  #
  ## LoadBalancer IP if service.type is LoadBalancer
  ## Default: nil
  ##
  # loadBalancerIP: 10.2.2.2
  annotations: {}
    # Annotation example: setup ssl with aws cert when service.type is LoadBalancer
    # service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-east-1:EXAMPLE_CERT
  labels: {}
    ## Label example: show service URL in `kubectl cluster-info`
    # kubernetes.io/cluster-service: "true"
  ## Limit load balancer source ips to list of CIDRs (where available)
  # loadBalancerSourceRanges: []
  selector: {}

ingress:
  enabled: false
  # hosts:
    # - kibana.localhost.localdomain
    # - localhost.localdomain/kibana
  # annotations:
  #   kubernetes.io/ingress.class: nginx
  #   kubernetes.io/tls-acme: "true"
  # tls:
    # - secretName: chart-example-tls
    #   hosts:
    #     - chart-example.local

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  # If set and create is false, the service account must be existing
  name:

livenessProbe:
  enabled: false
  path: /status
  initialDelaySeconds: 30
  timeoutSeconds: 10

readinessProbe:
  enabled: false
  path: /status
  initialDelaySeconds: 30
  timeoutSeconds: 10
  periodSeconds: 10
  successThreshold: 5

# Enable an authproxy. Specify container in extraContainers
authProxyEnabled: false

extraContainers: |
# - name: proxy
#   image: quay.io/gambol99/keycloak-proxy:latest
#   args:
#     - --resource=uri=/*
#     - --discovery-url=https://discovery-url
#     - --client-id=client
#     - --client-secret=secret
#     - --listen=0.0.0.0:5602
#     - --upstream-url=http://127.0.0.1:5601
#   ports:
#     - name: web
#       containerPort: 9090

extraVolumeMounts: []

extraVolumes: []

resources: {}
  # limits:
  #   cpu: 100m
  #   memory: 300Mi
  # requests:
  #   cpu: 100m
  #   memory: 300Mi

priorityClassName: ""

# Affinity for pod assignment
# Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
# affinity: {}

# Tolerations for pod assignment
# Ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
tolerations: []

# Node labels for pod assignment
# Ref: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}

podAnnotations: {}
replicaCount: 1
revisionHistoryLimit: 3

# Custom labels for pod assignment
podLabels: {}

# To export a dashboard from a running Kibana 6.3.x use:
# curl --user <username>:<password> -XGET https://kibana.yourdomain.com:5601/api/kibana/dashboards/export?dashboard=<some-dashboard-uuid> > my-dashboard.json
# A dashboard is defined by a name and a string with the json payload or the download url
dashboardImport:
  enabled: false
  timeout: 60
  xpackauth:
    enabled: true
    username: myuser
    password: mypass
  dashboards: {}
    # k8s: https://raw.githubusercontent.com/monotek/kibana-dashboards/master/k8s-fluentd-elasticsearch.json

# List of plugins to install using initContainer
# NOTE : We notice that lower resource constraints given to the chart + plugins are likely not going to work well.
plugins:
  # set to true to enable plugins installation
  enabled: false
  # set to true to remove all kibana plugins before installation
  reset: false
  # Use <plugin_name,version,url> to add/upgrade plugin
  values:
    # - elastalert-kibana-plugin,1.0.1,https://github.com/bitsensor/elastalert-kibana-plugin/releases/download/1.0.1/elastalert-kibana-plugin-1.0.1-6.4.2.zip
    # - logtrail,0.1.31,https://github.com/sivasamyk/logtrail/releases/download/v0.1.31/logtrail-6.6.0-0.1.31.zip
    # - other_plugin

persistentVolumeClaim:
  # set to true to use pvc
  enabled: false
  # set to true to use you own pvc
  existingClaim: false
  annotations: {}

  accessModes:
    - ReadWriteOnce
  size: "5Gi"
  ## If defined, storageClassName: <storageClass>
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
  ##   GKE, AWS & OpenStack)
  ##
  # storageClass: "-"

# default security context
securityContext:
  enabled: false
  allowPrivilegeEscalation: false
  runAsUser: 1000
  fsGroup: 2000

extraConfigMapMounts: []
  # - name: logtrail-configs
  #   configMap: kibana-logtrail
  #   mountPath: /usr/share/kibana/plugins/logtrail/logtrail.json
  #   subPath: logtrail.json

# Add your own init container or uncomment and modify the given example.
initContainers: {}
  ## Don't start kibana till Elasticsearch is reachable.
  ## Ensure that it is available at http://elasticsearch:9200
  ##
  # es-check:  # <- will be used as container name
  #   image: "appropriate/curl:latest"
  #   imagePullPolicy: "IfNotPresent"
  #   command:
  #     - "/bin/sh"
  #     - "-c"
  #     - |
  #       is_down=true
  #       while "$is_down"; do
  #         if curl -sSf --fail-early --connect-timeout 5 http://elasticsearch:9200; then
  #           is_down=false
  #         else
  #           sleep 5
  #         fi
  #       done

  关于kibana的配置就比较简单了，主要就是：

  1. 镜像换成私有仓库地址；
  2. kibana的配置文件中配置连接ES的地址，注意是连接elasticsearch-client的9200端口；
  3. kibana就部署一个deployment无状态应用就行。前面提到的ES就一定要是StatefulSet资源类型,fluentd一定要是DaemonSet资源类型；
  4. 可以定义一个ingress,因为kibana才是给用户提供访问的；
    kubernetes EFK日志管理系统
    kubernetes EFK日志管理系统
    kubernetes EFK日志管理系统
    kubernetes EFK日志管理系统
    kubernetes EFK日志管理系统
    kubernetes EFK日志管理系统
    kubernetes EFK日志管理系统
    kubernetes EFK日志管理系统

推荐关注我的个人微信公众号 “云时代IT运维”,周期性更新最新的应用运维类技术文档。关注虚拟化和容器技术、CI/CD、自动化运维等最新前沿运维技术和趋势；

kubernetes EFK日志管理系统

转载于:https://blog.51cto.com/zgui2000/2388379

weixin_34387468
关注
Kubernetes EFK
weixin_45735613的博客
03-18 239
Kubernetes EFK Elasticsearch + Fluentd + kibana 的构建： Elasticsearch-rc 配置文件： apiVersion: v1 kind: ReplicationController metadata: name: elasticsearch-logging-v1 namespace: kube-system labels: k8s-app: elasticsearch-l...
Kubernetes 基于 EFK 技术栈的日志收集实践
qhh0205
09-06 4859
之前写过一篇文章介绍了容器环境下日志管理的原理机制：从 Docker 到 Kubernetes 日志管理机制详解，文章内容偏理论，本文在该理论的支撑下具体实践 Kubernetes 下基于 EFK 技术栈的日志收集，本文偏实践，要想全面了解 Kubernetes 下日志收集管理机制，最好还是两篇文章顺序阅读。 本文不仅限于介绍怎么在 Kubernetes 集群部署 EFK 组件，还涉及到其他相关话...
kubernetes-efk：设置Elasticsearch，Fluentd和Kibana（EFK）
02-22
Kubernetes上的弹性云 注意：Elasticstack中的命名空间已更改为logging 在此存储库中，我们将设置Elasticsearch和Kibana，然后设置Fluentd 松紧带 在本节中，我们将与Elasticsearch和Kibana一起设置elastitc运算符 在您的Kubernetes集群中部署ECK 安装自定义资源定义和操作员及其RBAC规则： kubectl apply -f elastic/all-in-one.yaml kubectl -n logging logs -f statefulset.apps/elastic-operator监视操作员日志： 部署Elasticsearch集群 应用简单的Elasticsearch集群规范，其中包含三个Elasticsearch节点： 部署kubectl apply -f elastic/elast
在kubernetes集群中部署efk
weixin_33713503的博客
02-20 316
本文将介绍在kubernetes 1.9集群下配置elasticsearch、fluentd、kibana集中收集k8s集群日志信息。俗称EFK，其中elasticsearch负责存储日志。fluentd负责将集群中docker主机上的日志发送给elasticsearch，因此fluentd在k8s集群中需要以daemonset的方式运行。kibana负责图形化展示日志信息。 一、环境介绍软件环境...
K8S日志收集EFK
henanchenxuyuan的博客
09-03 1430
基于云原生 12 要素开发的程序，一般会将日志直接输出到控制台，并非是指定一个文件存储日志,这一点和传统架构还是有区别的。但是公司内的程序并非都是基于云原生要素开发的，比如一些年代已久的程序，这类程序如果部署至 Kubernetes 集群中，就需要考虑如何将输出至本地文件的日志采集到Elasticsearch。
kubernetes-EFK
小飞
09-16 265
EFK 主流的 ELK（Elasticsearch，Logstash，Kibana）目前已经转变为 EFK（Elasticsearch，Filebeat or Fluentd，Kibana），对于容器云的日志方案业内也普遍推荐采用 Fluentd logstash支持所有主流日志类型，插件支持最丰富，可以灵活DIY，但性能较差，JVM容易导致内存使用量高。 fluentd支持所有主流日志类型，插件支持较多，性能表现较好。 logtail占用机器cpu、内存资源最少，结合阿里云日志服务的E2E体验良
kubernetes（k8s）中部署 efk
小云的博客
12-28 1052
Kubernetes 开发了一个 Elasticsearch 附加组件来实现集群的日志管理。这是一个 Elasticsearch、Fluentd 和 Kibana 的组合。Elastics...
Kubernetes离线部署+EFK日志收集资源.rar
05-15
其次，**EFK日志收集**是Kubernetes环境中常见的日志管理和分析解决方案。EFK由三个组件组成： 1. **Elasticsearch**：是一个分布式、RESTful风格的搜索和数据分析引擎，用于存储、搜索和分析大量日志数据。 2. **...
Kubernetes+EFK构建日志分析平台
z2050025507的博客
08-17 1572
由于 Kubernetes 在容器编排领域的强势领先，使得越来越多的企业将业务迁至基于 Docker+Kubernetes 技术栈打造的容器管理平台，所以在 Kubernetes 集群环境下如何打造高效、可靠的业务日志收集系统也成为企业必须面临的问题。Kubeadm 通过初始化安装是不包括网络插件的，也就是说初始化之后是不具备相关网络功能的，比如 k8s-master 节点上查看节点信息都是“Not Ready”状态、Pod 的 CoreDNS无法提供服务等。Elasticsearch 具有如下特点。
K8s中EFK日志采集部署
最新发布
qianshang666的博客
09-11 1146
Kubernetes 集群作为一个整体，我们需要统一收集日志。但是 Kubernetes 并不提供任何日志收集功能，因此您需要设置一个集中的日志后端
kubernetes日志采集方案EFK
总结、分享、交流
06-28 2784
https://www.qikqiak.com/post/install-efk-stack-on-k8s/ EFK简介 Kubernetes中比较流行的日志收集解决方案是Elasticsearch、Fluentd 和 Kibana（EFK）技术栈，也是官方现在比较推荐的一种方案。 Elasticsearch是一个实时的、分布式的可扩展的搜索引擎，允许进行全文、结构化搜索，它通常用于索引和搜索大量...
基于kubernetes-1.23.1搭建EFK
热门推荐
大风车儿的博客
03-23 21万+
在k8s里搭建EFK日志收集组件
Kubernetes之EFK搭建详解
大美天地的博客
12-15 441
EFK搭建 在install-k8s目录下新建efk文件夹 mkdir -p efk cd efk 安装elasticsearch helm repo add incubator https://charts.helm.sh/incubator 创建efk命名空间 kubectl create namespace efk helm fetch incubator/elasticsearch helm install --name els1 --namespace=efk -f values.yaml i
Kubernetes 1.11.3上使用动态PV部署EFK 6.4.0
weixin_34309435的博客
11-28 163
Kubernetes 上部署 efk 日志收集系统 环境版本：OS：CentOS 7.4(1804)Kubernetes：1.11.3Elasticsearch：5.4.0Fuentd-elasticsearch：2.0.2Kibana：6.4.0 本地镜像仓库：192.168.22.8NFS-Server：192.168.22.8 部署顺序：Kibana -> NFS动态存储 -> ...
kubernetes日志系统，装efk
hxfzr1314521的博客
06-08 244
k8s日志组件安装 经过一段时间的赶工项目终于上线了，但是连日志系统都没有，终于今天抽空安装了下，过程非常简单。 1.日志组件的选择 看来看去还是选了官方推荐的fluentd+kibana+elasticsearch，因为项目中用到了es，是现成的，怎么简单怎么来。 2.安装 非常简单，抄github即可。这是git地址：https://github.com/kubernetes/kubernetes/tree/v1.16.4/cluster/addons/fluentd-elasticsearc
Kubernetes(1.6.2)部署 EFK 插件
weixin_33973609的博客
12-25 120
2019独角兽企业重金招聘Python工程师标准>>> ...
kubernetes部署efk-7.2.0
weixin_42603477的博客
04-28 276
kubernetes部署efk-7.2.0 1、首先准备好相关工具 2、创建命名空间 由于我yaml文件里用的命名空间都是kube-logging，所以先创建一个kube-logging的命名空间。 kubectl create ns kube-logging 3、将准备好的镜像上传到本地,需要所有节点都上传 docker load -i elasticsearch_7_2_0.tar.gz docker load -i fluentd.tar.gz docker load -i kibana_7_2_
Kubernetes-基于EFK进行统一的日志管理原理(kibana查询语法)
全栈工程师开发手册（原创）https://github.com/tencentmusic/cube-studio
11-01 4572
EFK安装部署参考:https://blog.csdn.net/luanpeng825485697/article/details/83312662 1、统一日志管理的整体方案 通过应用和系统日志可以了解Kubernetes集群内所发生的事情，对于调试问题和监视集群活动来说日志非常有用。对于大部分的应用来说，都会具有某种日志机制。因此，大多数容器引擎同样被设计成支持某种日志机制。对于容器化应用程序...
《kubernetes-1.8.0》10-addon-EFK
Hek's blog
11-27 2592
《kubernetes-1.8.0》10-addon-EFK 《kubernetes 1.8.0 测试环境安装部署》 时间：2017-11-27 一、环境准备加载所需镜像：详见 《01-环境准备》下载所需yaml：官方文档鉴于镜像版本，以下贴出对应的yaml：es-service.yaml$ mkdir ~/EFK $ cd ~/EFK$ cat > es-service.yaml <
Kubernetes 1.9 HA集群部署：Heapster、DNS、EFK与Dashboard安装指南
本文档详细介绍了如何在Kubernetes 1.9环境中进行堆ster、dashboard、DNS（kube-dns）和ELK（Elasticsearch, Logstash, Kibana）日志管理系统的安装与部署，同时涉及搭建私有Docker Harbor仓库以及高可用性集群的...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值