Logstash :是开源的服务器端数据处理管道,能够同时从多个来源采集数据,转换数据,然后将数据发送到您最喜欢的 “存储库” 中。(我们的存储库当然是 Elasticsearch。)
logstash-6.4.3 功能这么强大该如何搭建和测试呢?
下面就一步步来完成。
一、安装依赖
安装jdk1.8版本。
$ sudo yum -y install java-1.8.0-openjdk
二、安装logstash-6.4.3
2.1 下载logstash-6.4.3
$ mkdir ~/package
$ cd ~/package
$ wget https://artifacts.elastic.co/downloads/logstash/logstash-6.4.3.tar.gz
2.2 安装logstash-6.4.3
$ mkdir ~/soft
$ tar -xf logstash-6.4.3.tar.gz -C ~/soft/
$ cd ~/soft
$ mv logstash-6.4.3 logstash
三、简单测试logstash-6.4.3
3.1 命令行测试。
3.1.1 命令行启动测试logstash-6.4.3
$ cd ~/soft/logstash/
$ ./bin/logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'
如果启动成功会出现一下的提示语句
[2019-04-17T20:27:26,361][INFO ][logstash.agent ] Pipelines running {:count=>1,:running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-04-17T20:27:26,649][INFO ][logstash.agent ] Successfully started LogstashAPI endpoint {:port=>9600}
接着屏幕就等着你输入了,比如输入一个Hello World,会出现以下的提示语句。
Hello world
{
"@timestamp" => 2019-04-17T12:27:59.767Z,
"message" => "Hello world",
"host" => "elk",
"@version" => "1"
}
3.2 配置文件简单测试
编辑配置文件
$ cd ~/soft/logstash/
$ vim config/logstash.conf
input {
stdin { }
}
output {
stdout {
codec => rubydebug { }
}
elasticsearch {
hosts => ["0.0.0.0:9200"]
}
}
启动elasticsearch
进入elasticsearch安装目录启动elasticsearch
指定配置文件启动logstash
$ ./bin/logstash -f config/logstash.conf
同样命令行等着你输入指令
Hello world
{
"@timestamp" => 2019-04-17T12:41:43.292Z,
"message" => "Hello world",
"@version" => "1",
"host" => "elk"
}
但实际上,这个测试远不止如此。
打开另外一个终端,输入:
$ curl http://127.0.0.1:9200/_search?q=hello ; echo
你会看到下面一段
{"took":792,"timed_out":false,"_shards":{"total":30,"successful":30,"skipped":0,"failed":0},"hits":{"total":22,"max_score":0.9808292,"hits":[{"_index":"logstash-2019.04.17","_type":"doc","_id":"LUdRK2oBjPq7VWGZOK_h","_score":0.9808292,"_source":{"@timestamp":"2019-04-17T12:41:43.292Z","message":"Hello world","@version":"1","host":"elk"}},{"_index":"logstash-2019.04.16","_type":"doc","_id":"JHlbJGoBWkDca1n7frqB","_score":0.2876821,"_source":{"message":"hello","host":"elk","@version":"1","@timestamp":"2019-04-16T04:15:33.952Z"}},{"_index":"logstash-2019.04.13","_type":"doc","_id":"ZHTRFWoBWkDca1n7hVSZ","_score":0.24116206,"_source":{"@timestamp":"2019-04-13T08:30:13.787Z","host":"elk","message":"hello world","@version":"1"}},{"_index":"logstash-2019.04.13","_type":"doc","_id":"snTfFWoBWkDca1n7ZFiB","_score":0.24116206,"_source":{"host":"elk","message":"hello world","@version":"1","@timestamp":"2019-04-13T08:45:22.838Z"}},{"_index":"logstash-2019.04.13","_type":"doc","_id":"t3TfFWoBWkDca1n7dFh7","_score":0.24116206,"_source":{"host":"elk","message":"hello world","@version":"1","@timestamp":"2019-04-13T08:45:26.926Z"}},{"_index":"logstash-2019.04.13","_type":"doc","_id":"uHTfFWoBWkDca1n7d1id","_score":0.24116206,"_source":{"host":"elk","message":"hello world","@version":"1","@timestamp":"2019-04-13T08:45:27.725Z"}},{"_index":"logstash-2019.04.13","_type":"doc","_id":"DXTgFWoBWkDca1n7dlmA","_score":0.24116206,"_source":{"host":"elk","message":"hello world","@version":"1","@timestamp":"2019-04-13T08:46:32.973Z"}},{"_index":"logstash-2019.04.13","_type":"doc","_id":"V3TRFWoBWkDca1n7VVRw","_score":0.18232156,"_source":{"@timestamp":"2019-04-13T08:30:00.993Z","host":"elk","message":"hello world","@version":"1"}},{"_index":"logstash-2019.04.13","_type":"doc","_id":"tHTfFWoBWkDca1n7alhI","_score":0.18232156,"_source":{"host":"elk","message":"hello world","@version":"1","@timestamp":"2019-04-13T08:45:24.318Z"}},{"_index":"logstash-2019.04.13","_type":"doc","_id":"J3TdFWoBWkDca1n7rVjY","_score":0.13353139,"_source":{"host":"elk","message":"hello world","@version":"1","@timestamp":"2019-04-13T08:43:30.031Z"}}]}}