efk测试环境部署

0.前言

• 测试环境部署的efk是没有数据持久化的，因此只能用于测试环境，如果部署到生产环境则需要部署storageclass或者使用别的方法实现数据持久化即可

1.es部署

$ kubectl create namespace efk
$ vim es.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: elasticsearch
  namespace: efk
spec:
  selector:
    matchLabels:
      component: elasticsearch
  template:
    metadata:
      labels:
        component: elasticsearch
    spec:
      containers:
        - name: elasticsearch
          image: elasticsearch:7.8.0
          imagePullPolicy: "IfNotPresent"
          env:
            - name: discovery.type
              value: single-node
          ports:
            - containerPort: 9200
              name: http
              protocol: TCP
          resources:
            limits:
              cpu: 500m
              memory: 2Gi
            requests:
              cpu: 500m
              memory: 2Gi
---
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  namespace: efk
  labels:
    service: elasticsearch
spec:
  type: NodePort
  selector:
    component: elasticsearch
  ports:
  - port: 9200
    targetPort: 9200
    nodePort: 31200

2.部署kibana

$ vim kibana.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana
  namespace: efk
spec:
  selector:
    matchLabels:
      run: kibana
  template:
    metadata:
      labels:
        run: kibana
    spec:
      containers:
        - name: kibana
          image: kibana:7.8.0
          imagePullPolicy: "IfNotPresent"
          env:
            - name: ELASTICSERACH_URL
              value: http://elasticsearch.efk.svc.cluster.local:9200      #对应elasticsearch节点的的服务地址或IP+端口号
            - name: XPACK_SECURITY_ENABLED
              value: "true"            
          ports:
            - containerPort: 5601
              name: http
              protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: kibana
  namespace: efk
  labels:
    service: kibana
spec:
  type: NodePort
  selector:
    run: kibana
  ports:
  - port: 5601
    targetPort: 5601
    nodePort: 31601

3.部署fluentd

$ vim fluentd-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: fluentd
  namespace: kube-system    #注意这一定要配置成k8s核心组件的名称空间（kube-system)
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: fluentd
  namespace: kube-system   #注意这一定要配置成k8s核心组件的名称空间（kube-system)
rules:
  - apiGroups:
      - ""
    resources:
      - pods
      - namespaces
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: fluentd
roleRef:
  kind: ClusterRole
  name: fluentd
  apiGroup: rbac.authorization.k8s.io
subjects:
  - kind: ServiceAccount
    name: fluentd
    namespace: kube-system    #注意这一定要配置成k8s核心组件的名称空间（kube-system)

$ fluentd-deployment.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system    #注意这一定要配置成k8s核心组件的名称空间（kube-system)
  labels:
    k8s-app: fluentd-efk
    version: v1
    kubernetes.io/cluster-service: "true"
spec:
  selector:
    matchLabels:
      k8s-app: fluentd-efk
      version: v1
  template:
    metadata:
      labels:
        k8s-app: fluentd-efk
        version: v1
        kubernetes.io/cluster-service: "true"
    spec:
      serviceAccount: fluentd
      serviceAccountName: fluentd
      tolerations:
        - key:  "node-role.kubernetes.io/master"
          effect: "NoSchedule"
      containers:
        - name: fluentd
          image: fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch
          imagePullPolicy: "IfNotPresent"
          env:
            - name: FLUENT_ELASTICSEARCH_HOST
              value: "elasticsearch.efk.svc.cluster.local"    #对应elasticsearch节点的的服务地址或IP
            - name: FLUENT_ELASTICSEARCH_PORT
              value: "9200"
            - name: FLUENT_ELASTICSEARCH_SCHEME
              value: "http"
            - name: FLUENT_UID
              value: "0" 
            - name: FLUENT_SYSTEMD_CONF
              value: disable   
          resources:
            limits:
              cpu: 200m
              memory: 200Mi
            requests:
              cpu: 200m
              memory: 100Mi
          volumeMounts:
            - name: varlog
              mountPath: /var/log
            - name: varlibdockercontainers
              mountPath: /data/docker/containers     # 对应docker在node节点上的日志目录，如安装Docker时没有改目录的话，那日志默认存放在/var/log/docker/containers目录；
              readOnly: true
      terminationGracePeriodSeconds: 30 
      volumes:
        - name: varlog
          hostPath:
            path: /var/log
        - name: varlibdockercontainers
          hostPath:
            path: /data/docker/containers    # 对应docker在node节点上的日志目录，如安装Docker时没有改目录的话，那日志默认存放在/var/log/docker/containers目录；

4.查看es

• 查看es的目的是为了查看日志是否发送到了es数据库上；
• edge浏览器点击扩展—获取Microsoft-Edge扩展—在搜索栏输入Elasticvue—点击安装即可
• 添加es集群随后就能看到日志索引了

$ kubectl port-forward -n efk statefulsets/es-cluster --address 0.0.0.0 9200