ELK单机测试--nginx json格式

最新推荐文章于 2024-08-29 21:10:03 发布

SoLucky2017

最新推荐文章于 2024-08-29 21:10:03 发布

阅读量344

点赞数

分类专栏： ELK

本文链接：https://blog.csdn.net/ssllkkyyaa/article/details/87866039

版权

ELK 专栏收录该内容

6 篇文章 0 订阅

订阅专栏

本文详细介绍如何使用ELK堆栈进行日志分析，重点讲解Logstash配置难点，包括JSON格式的日志收集、地理信息解析及Elasticsearch索引设置。提供从Nginx日志到Elasticsearch的完整配置流程。

摘要由CSDN通过智能技术生成

之前发了一个复杂版本的，今天发一个json格式的

ELK最大的难度就是logstash的启动配置文件

各种格式的不适应，总是conf报错

ELK 5.1.1

单机10.130.24.111

---------开发内网
nginx
配置
/usr/local/nginx/conf/nginx.conf

nginx stop:
pkill nginx
/usr/local/nginx/sbin/nginx -s stop

nginx log:
/usr/local/nginx/logs
tail -f /usr/local/nginx/logs/access.log
echo "">/usr/local/nginx/logs/access.log

log_format log_json '{"@timestamp":"$time_iso8601",'
               '"host":"$server_addr",'
               '"clientip":"$remote_addr",'
               '"size":$body_bytes_sent,'
               '"responsetime":$request_time,'
               '"upstreamtime":"$upstream_response_time",'
               '"upstreamhost":"$upstream_addr",'
               '"http_host":"$host",'
               '"url":"$uri",'
               '"xff":"$http_x_forwarded_for",'
               '"referer":"$http_referer",'
               '"agent":"$http_user_agent",'
               '"status":"$status"}';

access_log logs/access_nginx.json log_json;

具体文件见：

链接：https://pan.baidu.com/s/1rnJwuxxadvw9Mzc5EP8TAg
提取码：ksk2
复制这段内容后打开百度网盘手机App，操作更方便哦

---------------------

logstash

input {
   file {
       type => "nginx-access"
       path => "/usr/local/nginx/logs/access_nginx.json"
       start_position => "beginning"
       codec => "json"
   }
}
filter {
   if [type] == "nginx-access"{
       geoip {
           source => "clientip"
           target => "geoip"
           database => "/home/elasticresearch/soft/logstash-5.1.1/GeoLite2-City.mmdb"
           add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}"]
           add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}"]
       }
   }
}
output {
   if [type] == "nginx-access" {
       stdout{
           codec=>rubydebug
       }
       elasticsearch {
           action => "index"
           hosts  => "10.130.24.111:9200"
           index  => "logstash-slow-%{+yyyy.MM.dd}"
       }
   }
}