网上学习资料一大堆,但如果学到的知识不成体系,遇到问题时只是浅尝辄止,不再深入研究,那么很难做到真正的技术提升。
一个人可以走的很快,但一群人才能走的更远!不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人,都欢迎加入我们的的圈子(技术交流、学习资源、职场吐槽、大厂内推、面试辅导),让我们一起学习成长!
filebeat-配置文件:
filebeat.inputs:
- type: log
paths:
- /home/yanyaofeng/iperf/logs/*.log
tags: ["tupu_log", "iperf3_host"]
json.message_key: levelname
include_lines: ['info']
json.add_error_key: true
json.keys_under_root: true
output.logstash:
hosts: ["172.26.2.85:31415", "172.26.2.86:31415", "172.26.2.87:31415", "172.26.2.88:31415", "172.26.2.89:31415"]
#output.elasticsearch:
# hosts: ["172.26.2.85:39202", "172.26.2.86:39202", "172.26.2.87:39202", "172.26.2.88:39202", "172.26.2.89:39202"]
# index: "tupu-cs-host-iperfs-index-%{+YYYY.MM.dd}"
setup.template.name: "filebeattest"
setup.template.pattern: "filebeattest-*"
logstash-配置文件:
[root@gz-cs-2-88 conf.d]# cat tupu-log-pipeline.conf
##################################
# 测试执行命令:
# /usr/share/logstash/bin/logstash -f tupu-log-pipeline.conf --config.reload.automatic
##################################
input {
beats {
port => 31415
}
}
filter {
if "tupu_log" in [tags] and [tags][1] {
mutate {
add_field => { "log_project_name" => "%{[tags][1]}" }
}
}
if [tags][1] == "tupu-nginx" {
mutate {
convert => [ "status","integer" ]
convert => [ "size","integer" ]
convert => [ "upstreamtime","float" ]
remove_field => "message"
}
geoip {
source => "ip"
target => "ipgeo"
}
}
}
output {
if "tupu_log" in [tags] and [log_project_name] {
# stdout { codec => rubydebug }
elasticsearch {
hosts => ["172.26.2.85:39202", "172.26.2.86:39202", "172.26.2.87:39202", "172.26.2.88:39202", "172.26.2.89:39202"]
index => "tupu-log-%{log_project_name}-%{+YYYY.MM.dd}"
}
}
}
[root@gz-cs-2-88 logstash]#
[root@gz-cs-2-88 logstash]# cat pipelines.yml
# This file is where you define your pipelines. You can define multiple.
# For more information on multiple pipelines, see the documentation:
# https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html
- pipeline.id: main
path.config: "/etc/logstash/conf.d/*.conf"
[root@gz-cs-2-88 logstash]# cat logstash.yml | egrep -v '^#|^$|#'
path.data: /var/lib/logstash
pipeline.ecs_compatibility: disabled
path.logs: /var/log/logstash
es-配置:
[root@gz-cs-2-88 elasticsearch]# cat elasticsearch.yml | egrep -v '^#|^$|#'
cluster.name: log-es-production-new-1
node.name: es-chaosuan_2.88
node.roles: [ "master", "data" ]
xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
xpack.security.http.ssl.enabled: false
ingest.geoip.downloader.enabled: false
![img](https://img-blog.csdnimg.cn/img_convert/5f7ec6965aad6a01c98f6a7065d63982.png)
![img](https://img-blog.csdnimg.cn/img_convert/63df67d4294e35c585a31054fa03ed67.png)
**网上学习资料一大堆,但如果学到的知识不成体系,遇到问题时只是浅尝辄止,不再深入研究,那么很难做到真正的技术提升。**
**[需要这份系统化的资料的朋友,可以戳这里获取](https://bbs.csdn.net/topics/618631832)**
**一个人可以走的很快,但一群人才能走的更远!不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人,都欢迎加入我们的的圈子(技术交流、学习资源、职场吐槽、大厂内推、面试辅导),让我们一起学习成长!**
s/618631832)**
**一个人可以走的很快,但一群人才能走的更远!不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人,都欢迎加入我们的的圈子(技术交流、学习资源、职场吐槽、大厂内推、面试辅导),让我们一起学习成长!**