logstash

vim config/nginx.conf

	input {
	
	   file  {
	            path => "/usr/local/nginx/logs/access.log"
	            sincedb_write_interval => 5  # 每5秒钟写一次文件读取记录
	            start_position => "beginning" # 每个文件都从头开始读写
	    }
	}
	filter {
	    grok {
	
	        match => { "message" => "%{NGINXACCESS}" }
	    }
	
	}
	output {
	  elasticsearch {
	        hosts => ["localhost:17528"]
	        index => "parsed-nginx-access-log"
	  }
	  stdout { codec => rubydebug }
	}

这份配置文件里面用到的正则匹配规则NGINXACCESS，存放在patterns文件夹中，内容如下：

NGINXACCESS %{IPORHOST:remote_addr} - %{USERNAME:remote_user} \[%{HTTPDATE:time_local}\] "%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:http_version}" %{INT:status} %{INT:body_bytes_sent} %{QS:request_body} %{QS:http_referer} %{QS:http_user_agent} %{QS:http_x_forwarded_for} %{NUMBER:request_time:float} %{NUMBER:upstream_response_time:float}

启动 bin/logstash -f config/mynginx.conf