ERROR logstash/async.go:256 Failed to publish events caused by: write tcp 172.17.51.9:49746->10.0.3.104:5044: i/o timeout
ERROR pipeline/output.go:121 Failed to publish events: write tcp 172.17.51.9:49746->10.0.3.104:5044: i/o timeout
在部署K8s的ELK日志方案时,出现上述问题,且在kibana界面看不到索引模板index pattern。
kubectl get pods -n kube-system
-------
k8s-logs-kbfj4 1/1 Running 0 10s
k8s-logs-npqsf 1/1 Running 0 10s
-------
kubectl logs k8s-logs-kbfj4 -n kube-system
问题分析:
参见:https://github.com/elastic/logstash/issues/6279
由于在logstash配置中有多个输入的配置,导致的这个问题,在配置的logstash服务器中,有多个logstash进程,可用以下方式实现
ps -ef |grep logstash
kill -9 ID(多个进程)
运行
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash-to-es.conf --path.data=/root/data01
当该命令运行不起来时,可能是默认的data存储文件有.lock文件,删除即可
ls -alh
rm .lock
可在kibana界面中看到k8s-log日志。