Elasticsearch 插件
此附加组件由 Elasticsearch、Fluentd 和 Kibana 组合而成。 Elasticsearch 是一个搜索引擎,负责存储我们的日志并允许查询它们。 Fluentd 将日志消息从 Kubernetes 发送到 Elasticsearch,而 Kibana 是一个图形界面,用于查看和查询存储在 Elasticsearch 中的日志。
#1. 创建命名空间
创建logging-ns.yaml
kind: Namespace
apiVersion: v1
metadata:
name: logging
labels:
k8s-app: logging
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
应用yaml文件
$ kubectl apply -f logging-ns.yaml
#2. 部署Elasticsearch
添加chart仓库
$ helm repo add elastic https://helm.elastic.co
$ helm repo update
$ helm pull elastic/elasticsearch --version 7.5.2
$ tar -xvf elasticsearch-7.5.2.tgz && cd elasticsearch
安装
$ helm install elasticsearch . -n logging -f ./values.yaml
查看pv、pvc
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLAS S REASON AGE
pvc-23431c9b-d017-490b-946d-4fb4a3a14a2f 10Gi RWO Delete Bound logging/elasticsearch-master-elasticsearch-master-0 managed-nfs -storage 99s
$ kubectl get pvc -n logging
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
elasticsearch-master-elasticsearch-master-0 Bound pvc-23431c9b-d017-490b-946d-4fb4a3a14a2f 10Gi RWO managed-nfs-storage 106s
查看pod状态
$ kubectl get pods -n logging
NAME READY STATUS RESTARTS AGE
elasticsearch-master-0 1/1 Running 0 2m43s
查看service
$ kubectl get svc -n logging
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
elasticsearch-master ClusterIP 10.1.185.112 <none> 9200/TCP,9300/TCP 9m44s
elasticsearch-master-headless ClusterIP None <none> 9200/TCP,9300/TCP 9m44s
#3. 部署fluentd
下载chart
$ helm pull incubator/fluentd-elasticsearch --version 0.7.1
$ tar -xvf fluentd-elasticsearch-0.7.1.tgz && cd fluentd-elasticsearch
修改values.yaml
image:
repository: quay.io/fluentd_elasticsearch/fluentd
tag: v3.1.0
pullPolicy: IfNotPresent
elasticsearch:
host: 'elasticsearch-master-0'
port: 9200
安装
$ helm install fluentd . -n logging -f ./values.yaml
查看pod状态
$ kubectl get pods -n logging
NAME READY STATUS RESTARTS AGE
elasticsearch-master-0 1/1 Running 0 62m
fluentd-fluentd-elasticsearch-dgjc7 1/1 Running 0 111s
fluentd-fluentd-elasticsearch-hhgpr 1/1 Running 0 110s
查看configmap
$ kubectl get configmaps -n logging
NAME DATA AGE
fluentd-fluentd-elasticsearch 6 34m
#4. 部署kibana
下载chart
$ helm pull elastic/kibana --version 7.5.2
$ tar -xvf kibana-7.5.2.tgz && cd kibana
修改values.yaml
elasticsearchURL: "http://elasticsearch-master:9200"
elasticsearchHosts: "http://elasticsearch-master:9200"
kibanaConfig:
kibana.yml: |
server.name: kibana
server.host: "0.0.0.0"
elasticsearch.hosts: [ "http://elasticsearch-master:9200" ]
service:
type: LoadBalancer
loadBalancerSourceRanges:
- 192.168.238.0/24
ingress:
enabled: true
hosts:
- kibana.demo.com
安装
$ helm install kibana . -n logging -f ./values.yaml
查看pod状态
$ kubectl get pod -n logging
NAME READY STATUS RESTARTS AGE
elasticsearch-master-0 1/1 Running 1 4d18h
fluentd-fluentd-elasticsearch-dgjc7 1/1 Running 17 4d17h
fluentd-fluentd-elasticsearch-hhgpr 1/1 Running 17 4d17h
kibana-kibana-86866c888-8z2c5 1/1 Running 0 10m
查看ingress
$ kubectl get ingress -n logging
NAME CLASS HOSTS ADDRESS PORTS AGE
kibana-kibana <none> kibana.demo.com 192.168.238.156 80 4d16h
访问kibana,设置本地host解析,kibana.demo.com
#5. 添加索引
Fluentd 配置文件中采集的日志使用的是 logstash 格式,只需要输入logstash-*即可匹配到 Elasticsearch 集群中的所有日志数据。
其他对索引个性化需求的,可以自行配置。
参考文档:
[1]: https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch