ELK(15)：ELK显示Nginx日志中访客ip地理位置

 

ELK(15)：ELK显示Nginx日志中访客ip地理位置

 

https://www.cnblogs.com/ahaii/p/7410421.html

https://www.cnblogs.com/dance-walter/p/10144804.html

 

本文采用的是普通方式

pipline+es插件方式请移步

https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-geoip.html

1             环境准备：

1. ELK stack 环境一套 参考前面的吧
2. Nginx日志处理成json类型 ELK(6)：ELK-logstash收集Nginx日志
3. Nginx日志要获取用户真实ip地址nginx获取客户端真实ip
4. geolite数据库文件   geolite官网： https://dev.maxmind.com
5. index必须是logstash-*
6. 本文基于kibana7.2，把kibana设成英文界面，简体中文有bug
   

                  /etc/kibana/kibana.yml
                   i18n.locale: "en"
   

7. 高德地图的wmsAPI已经失效了，采用elasticsearch自带的地图就行了

 

2     下载geolite数据库(logstash机器上解压,logstash需调用)：

wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz 
[admin@pe-jira logstash]$ pwd
/etc/logstash
[admin@pe-jira logstash]$ ls
conf.d  GeoLite2-City  jvm.options  log4j2.properties  logstash-sample.conf  logstash.yml  patterns  pipelines.yml  startup.options
[admin@pe-jira logstash]$ tree GeoLite2-City/
GeoLite2-City/
├── COPYRIGHT.txt
├── GeoLite2-City.mmdb
├── LICENSE.txt
└── README.txt

 

 

3     logstash配置

日志格式

{"@timestamp":"2019-07-25T12:55:39+08:00","host":"10.6.76.27","request_method": "POST", "clientip":"1.80.80.50","size":470,"responsetime":0.013,"upstreamtime":"0.013","upstreamhost":"10.6.76.27:5601","http_host":"kibana.corp.zhaonongzi.com","url":"/elasticsearch/_msearch","xff":"1.80.80.50","referer":"http://kibana.corp.zhaonongzi.com/app/kibana","agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:68.0) Gecko/20100101 Firefox/68.0","status":"200"}

 

logstash配置

[admin@pe-jira conf.d]$ pwd
/etc/logstash/conf.d
[admin@pe-jira conf.d]$ cat kibana.conf
input {
    file {
        type => "pe-jira-kibana"
        path => "/home/admin/webserver/logs/kibana.log"
        start_position => "beginning"
        stat_interval => "2"
        codec => "json"
    }
}

filter {
 if[type] == "pe-jira-kibana" {
 grok {
        match => {
            "message" => "%{IPORHOST:clientip}
%{NOTSPACE:http_name} \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response}(?:%{NUMBER:bytes:float}|-) %{QS:referrer} %{QS:agent}"
        }
    }
    geoip {
      source => "clientip"
      target => "geoip"
      database => "/etc/logstash/GeoLite2-City/GeoLite2-City.mmdb"
      add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
      add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
    }

    mutate {
      convert => [ "[geoip][coordinates]", "float" ]
      convert => [ "response","integer" ]
      convert => [ "bytes","integer" ]
      replace => { "type" => "pe-jira-kibana" }
      remove_field => "message"
    }

    date {
      match => [ "timestamp","dd/MMM/yyyy:HH:mm:ss Z"]

    }
    mutate {
      remove_field => "timestamp"

    }
}
}
output {
  if [type] == "pe-jira-kibana" {
    elasticsearch {
      hosts => ["10.6.76.27:9200"]
      index => "logstash-pe-jira-nginx-kibana-%{+YYYY.MM.dd}"
      }
  }

}

 

 

 

配置解释

• geoip：IP查询插件，默认安装
• source: 需要通过geoip插件处理的filed，根据实际情况中nginx日志ip的field_name来修改。
• target: 解析后的geoip地址数据，存放在定义的字段中，默认geoip这个字段
• database: 指定下载的数据库文件
• add_field: 添加经纬度，地图的定位显示是依靠经纬度来实别的

检查配置重启

 

4     kibana配置

查看geoip是否取出

 

 

 配置地图

 

转载于:https://www.cnblogs.com/wangxu01/articles/11244570.html