logstash 配置

wq_14365327

已于 2022-09-06 14:08:07 修改

阅读量476

点赞数

分类专栏： Kibana + ES + logstash 文章标签：后端中间件

于 2022-08-02 17:16:59 首次发布

本文链接：https://blog.csdn.net/wq_14365327/article/details/126125897

版权

Kibana + ES + logstash 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

1. logstash 启动运行

bin/logstash -f config/logstash_nginx_filebeat.conf

2. logstash_nginx_filebeat.conf 是编写进行配置


input {
    beats {
        # filebeat 链接端口
	    port => "5044"
        # 链接超时
    	client_inactivity_timeout => 36000
    }
}


filter {
    ruby {
	    init => "@kname =['remote_addr','time_local','request_method','status']"
	    code => "new_event = LogStash::Event.new(Hash[@kname.zip(event.get('message').split('|'))])
                new_event.remove('@timestamp')
                event.append(new_event)
                "
    }
    geoip {
        source => "remote_addr"
        target => "geoip"
        database => "/opt/GeoLite2-City_20190716/GeoLite2-City.mmdb"
    }
    date {
    	locale => "en"
    	match => [ "time_local", "dd/MMM/yyyy:HH:mm:ss Z" ]
    }
    mutate {
	    convert => [
            "status","integer"
        ]
    }
}


output {
    elasticsearch {
        hosts => ["localhost:9200"]
    	#自定义索引名,通过这个索引在kibana里面搜索并创建查看信息
    	index => "logstash-nginx-%{+YYYY.MM.dd}"
    }
}

logstash 重启后的信息：

Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}

index => "logstash-nginx-%{+YYYY.MM.dd}" 对应 es 中添加的模板 template 的 index_patterns，默认状态配置；

index_patterns 对应logstash配置 output -> elasticsearch -> index；

如果 logstash配置中的index 修改，那么 es 中对就的模板也要进行修改；