Windows环境下的ELK——收集apache标准日志（3）

一、运行EK

环境是基于ELK7.5.0版本的配置

1.打开elasticsearch

访问http://ip:9200/
出现如下图，运行成功

2.运行Kibana

出现如下图，则运行成功

二、设置配置文件

1.配置conf文件

与bin同级目录

2.配置标准输入到ES

input{
stdin{}
}
output{
        elasticsearch{
                hosts=>["119.3.20.91:9200"]
                index => "test2-%{+YYYY.MM.dd}"
                }
        stdout{codec => rubydebug}
}

参数解释

1. hosts主机地址：端口
2. index 索引,对于本次日志文件建立的索引，方便分析，查询

三、控制台标准输入

1.cmd进入bin目录

2.执行conf文件

logstash.bat -f SSL.conf

3.输入消息

4.查看结果

4.1 创建索引

按步骤来，创建完成

4.2 查看数据

四、日志文件输入

1.文件目录

2.multi-input.conf

input {
  file {
    path => "F:/ELK/ELK8.2.3/logstash-8.2.3/apache.log"
  	start_position => "beginning"
  }
}



output {
	stdout {
		codec => rubydebug
	}

	  	elasticsearch {
		hosts=>["119.3.20.91:9200"]
	    	index => "apache_test1"
	    	template => "F:/ELK/ELK8.2.3/logstash-8.2.3/apache_template.json"
	    	template_name => "apache_elastic_example"
	    	template_overwrite => true
	  }	
}

1. template_name: 映射模板的名字,template_name如果不指定的话，会使用默认值logstash

2. template: 模板的路径

3. template_overwrite: 是否覆盖已存在的模板，template_overwrite为true则template的order高的，满足同样条件的template将覆盖order低的

4. 在配置多个数据模板时候,要多加一个配置项： template_name ，切该名字必须全部为小写

2.apace.log

83.149.9.216 - - [17/May/2015:10:05:03 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:43 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-dashboard3.png HTTP/1.1" 200 171717 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:47 +0000] "GET /presentations/logstash-monitorama-2013/plugin/highlight/highlight.js HTTP/1.1" 200 26185 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:12 +0000] "GET /presentations/logstash-monitorama-2013/plugin/zoom-js/zoom.js HTTP/1.1" 200 7697 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:07 +0000] "GET /presentations/logstash-monitorama-2013/plugin/notes/notes.js HTTP/1.1" 200 2892 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"

3.apache_template.json

{

  "template": "apache_elastic_example",
  "settings": {
     "index.refresh_interval": "5s"
  },
  "mappings": {
     "_default_": {
        "dynamic_templates": [
           {
              "message_field": {
                 "mapping": {
                    "norms": false,
                    "type": "text"
                 },
                 "match_mapping_type": "string",
                 "match": "message"
              }
           },
           {
              "string_fields": {
                 "mapping": {
                    "norms": false,
                    "type": "text",
                    "fields": {
                       "keyword": {
                          "ignore_above": 256,
                          "type": "keyword"
                       }
                    }
                 },
                 "match_mapping_type": "string",
                 "match": "*"
              }
           }
        ],
        "properties": {
           "geoip": {
              "dynamic": true,
              "properties": {
                 "location": {
                    "type": "geo_point"
                 },
                 "ip": {
                   "type": "ip"
                 },
                 "continent_code": {
                  "type": "keyword"
                 },
                 "country_name": {
                   "type": "keyword"
                 }
              },
              "type": "object"
           },
           "@version": {
              "type": "keyword"
           }
        }
     }
  }
}

4.执行命令

logstash.bat -f multi-input.conf

五、ELK8.2.3版本conf配置文件

因为ELK8.2.3默认开启了安全组件

input{
	stdin{
}
}
output{
        elasticsearch{
        #开启SSL，使用https进行访问
	ssl => true
                hosts=>["127.0.0.1:9200"]
                #用户
	user=>"elastic"
	#密码
	password=>"E*5Cu18yLj9*XDfnzti4"
	#设置索引
                index => "test1-%{+YYYY.MM.dd}"
                //取消安全证书的验证，如果为true那么即使用户密码正确也无法登录，还需要验证证书
	ssl_certificate_verification => false
                }
        stdout{codec => rubydebug}
}