【Python】windows10搭建ELK5日志收集系统

需要准备的软件

• ELK官网：https://www.elastic.co/cn/products
• JDK官网：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
  

---

安装elasticsearch-5.5.1

• 首先要配好JDK的环境变量，保存CMD可以运行Java和javac
• 安装好后访问：localhost:9200

• 打开谷歌浏览器，安装扩展程序：elasticsearch-head

---

安装logstash-5.5.1

• 解压文件，进入bin目录
• 创建一个配置文件，这里叫nginx_data.config

input {
    file {
        path => "C:\Users\DELL\Desktop\ELK\nginx.log"
        start_position => "beginning"
    }
}
filter {
    grok {
      match => { "message" => "%{IP:client} - - \[%{HTTPDATE:logdate}\] \"%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:http_status_code} %{NUMBER:bytes} \"%{NOTSPACE:ref}\" \"%{DATA:user_agent}\"" }
    }
    date {
        match => ["logdate", "dd/MMM/yyyy:HH:mm:ss Z"]
        target => "@timestamp"
    }
    kv {
                source => "request"
                field_split => "&?"
                value_split => "="
        }
    urldecode {
        all_fields => true
    }
}
output {
    elasticsearch {
        hosts => "localhost:9200"
    }
    stdout {
        codec => json_lines
    }
}

• nginx.log为：

127.0.0.1 - - [18/Aug/2017:01:00:07 +0800] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-"
127.0.0.1 - - [18/Aug/2017:01:00:07 +0800] "GET /favicon.ico HTTP/1.1" 404 571 "http://localhost/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-"
127.0.0.1 - - [18/Aug/2017:01:00:10 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-"
127.0.0.1 - - [18/Aug/2017:01:00:14 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-"
127.0.0.1 - - [18/Aug/2017:01:00:59 +0800] "GET /favicon.ico HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36" "-"
127.0.0.1 - - [18/Aug/2017:01:01:00 +0800] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36" "-"
127.0.0.1 - - [18/Aug/2017:01:01:00 +0800] "GET /favicon.ico HTTP/1.1" 404 571 "http://localhost/" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36" "-"
127.0.0.1 - - [18/Aug/2017:01:01:00 +0800] "GET /favicon.ico HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36" "-"

• 运行：logstash.bat -f nginx_data.config 往ES中导入数据

---

安装kibana-5.5.1-windows-x86

• 解压文件，修改conf/kibana.yml文件，解开连接ES的注释
• 进入bin目录，执行kibana.bat 即可启动
• 访问http://localhost:5601 进行配置