Windows下的ELK使用笔记

1.Elasticsearch

windows下运行es

• 1.进入bin目录
• 2.运行elasticsearch.bat
• 3.访问是否启动成功:http://localhost:9200?pretty

将ES作为windows后台服务

• 1.同样进入bin目录
• 2.执行 elasticsearch-service.bat install
• 3.访问是否启动成功:http://localhost:9200?pretty

ES与传统DB对比

• Relational DB -> Databases -> Tables -> Rows -> Columns
• Elasticsearch -> Indices -> Types -> Documents -> Fields
• es中的概念为索引(对应数据库),类型(对应表),文档(对应行),字段(对应列)

ES简单增删改查

• 参考文档 https://es.xiaoleilu.com/010_Intro/30_Tutorial_Search.html
• 创建索引

//创建一个索引
http://localhost:9200/test/user/1 put
{
	"userName":"cjh",
	"birthday":"1994-10-25",
	"createTime":"2019-02-28"
}
//返回创建成功
{
    "_index": "test",   //索引
    "_type": "user",    //类型
    "_id": "1",         //id
    "_version": 1,
    "result": "created",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 0,
    "_primary_term": 1
}

• 检索文档

//检索文档 /索引/类型/id
http://localhost:9200/test/user/1 get
//返回数据
{
    "_index": "test",
    "_type": "user",
    "_id": "1",
    "_version": 1,
    "_seq_no": 0,
    "_primary_term": 1,
    "found": true,
    "_source": {            //文档信息储存在_source中
        "userName": "cjh",
        "birthday": "1994-10-25",
        "createTime": "2019-02-28"
    }
}

• 简单搜索

//简单搜索 /索引/类型/_search
http://localhost:9200/test/user/_search
//返回结果
{
    "took": 80,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 1,
        "hits": [
            {
                "_index": "test",
                "_type": "user",
                "_id": "2",
                "_score": 1,
                "_source": {
                    "userName": "lmy",
                    "birthday": "1995-05-29",
                    "createTime": "2019-02-28"
                }
            },
            {
                "_index": "test",
                "_type": "user",
                "_id": "1",
                "_score": 1,
                "_source": {
                    "userName": "cjh",
                    "birthday": "1994-10-25",
                    "createTime": "2019-02-28"
                }
            }
        ]
    }
}

• 条件查询

//查询姓名为cjh的
http://localhost:9200/test/user/_search?q=userName:cjh
//使用DSL语句查询
http://localhost:9200/test/user/_search
{
    "query" : {
        "match" : {
            "userName" : "cjh"
        }
    }
}

• 更复杂的查询

//查询年龄大于20 姓名cjh
http://localhost:9200/test/user/_search
{
    "query" : {
        "filtered" : {
            "filter" : {
                "range" : {
                    "age" : { "gt" : 20 }   //gt为"greater than"的缩写
                }
            },
            "query" : {
                "match" : {
                    "userName" : "cjh" 
                }
            }
        }
    }
}

• 全文检索

http://localhost:9200/test/user/_search
{
    "query" : {
        "match" : { //使用match_phrase可以实现短语搜索 即该词语要相邻
            "about" : "rock climbing"
        }
    }
}

• 删除

POST {index}/_delete_by_query
{
    "query": {
        "match": {
            "message": "some message"
        }
    }
}

在版本2.X之后，delete_by_query被弃用，直接用delete就行：

DELETE {index}/{type}/_query
{
    "query": {
    "match_all": {}
    }
}

2.Kibana

windows下运行kibane

• 1.进入binmul
• 2.运行kibana.bat
• 3.访问5601端口验证是否启动成功:http://localhost:5601

简单使用

• 参考文章:https://www.cnblogs.com/cjsblog/p/9476813.html

3.Logstash

windows下使用logstash

• 1.bin目录下创建logstash.conf

input {
    stdin{
    }
} 
 
output {
    stdout{
    }
}

• 2.输入命令:logstash -f logstash.conf 启动成功!
• 3.输入hello world

hello world
{
      "@version"=>"1",
          "host"=>"cjh",
       "message"=>"hello world",
    "@timestamp"=>2019-03-02T09:01:33.272Z
}

• 4.logstash -f logstash.conf -- config.test_and_exit
  --config.test_and_exit选项的意思是解析配置文件并报告任何错误
• 5.logstash -f logstash.conf --config.reload.automatic
  --config.reload.automatic选项的意思是启用自动配置加载，以至于每次你修改完配置文件以后无需停止然后重启Logstash

与filebeat配合使用

• filebeat参考:https://www.cnblogs.com/cjsblog/p/9445792.html
• 1.配置filebeat.yml

filebeat.inputs:
- type: log
  paths:
    - D:\log\*.log //获取在D:\log\*.log路径下的所有文件作为输入，这就意味着Filebeat将获取/var/log目录下所有以.log结尾的文件

output.logstash:    //此处还可以直接指定输出到es
  hosts: ["localhost:5044"]

• 2.相应的,logstash的配置文件也要相应修改

input {
    beats{
		port => "5044"      //指定端口
    }
} 
 
output {
    stdout{
		codec => rubydebug  //这里是输出到控制台 也可以输出到es
    }
}

• 3.使用kv过滤
  kv形式过滤 title=a age=16 会将'='前后拆分成字段，默认是空格区分多个字段

input {
    beats{
		port => "5044"
    }
} 
filter{
	kv{}
} 
output {
    stdout { 
		codec => rubydebug
    }
}