CentOS下安装Logstash（附带示例）

最新推荐文章于 2023-11-22 13:44:31 发布

weixin_34409703

最新推荐文章于 2023-11-22 13:44:31 发布

阅读量190

点赞数

文章标签：数据库 java ruby

原文链接：https://my.oschina.net/wangmengjun/blog/861636

版权

2019独角兽企业重金招聘Python工程师标准>>>

本文记录一下在CentOS 6.7上，安装Logstash，版本为logstash-2.4.0.tar.gz。

Logstash是一个开源的日志管理工具。

下载安装包

使用wget命令下载logstash安装包，如

[root@dev18 srv]# wget https://download.elastic.co/logstash/logstash/logstash-2.4.0.tar.gz
--2017-03-17 16:37:14--  https://download.elastic.co/logstash/logstash/logstash-2.4.0.tar.gz
Resolving download.elastic.co... 107.22.208.105, 54.243.211.74, 107.21.249.70, ...
Connecting to download.elastic.co|107.22.208.105|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 83882952 (80M) [application/x-gzip]
Saving to: “logstash-2.4.0.tar.gz”

100%[====================================================================================================================================================================================================================================>] 83,882,952  7.90M/s   in 1m 54s  

2017-03-17 16:39:10 (721 KB/s) - “logstash-2.4.0.tar.gz” saved [83882952/83882952]

[root@dev18 srv]#

解压

使用tar -zvxf解压缩Logstash，如：

[root@dev18 srv]# tar -zvxf logstash-2.4.0

... ...

logstash-2.4.0/vendor/jruby/lib/ruby/shared/securerandom.rb
logstash-2.4.0/vendor/jruby/lib/ruby/shared/syslog.rb
logstash-2.4.0/vendor/jruby/lib/ruby/shared/tempfile.rb
logstash-2.4.0/vendor/jruby/lib/ruby/shared/tmpdir.rb
logstash-2.4.0/vendor/jruby/lib/ruby/shared/ubygems.rb
logstash-2.4.0/vendor/jruby/tool
logstash-2.4.0/vendor/jruby/tool/nailgun
logstash-2.4.0/vendor/jruby/tool/nailgun/Makefile.in
logstash-2.4.0/vendor/jruby/tool/nailgun/README.txt
logstash-2.4.0/vendor/jruby/tool/nailgun/configure
logstash-2.4.0/vendor/jruby/tool/nailgun/ng.exe
logstash-2.4.0/vendor/jruby/tool/nailgun/src
logstash-2.4.0/vendor/jruby/tool/nailgun/src/c
logstash-2.4.0/vendor/jruby/tool/nailgun/src/c/ng.c
logstash-2.4.0/vendor/bundle/jruby/1.9/gems/jrjackson-0.3.9-java/.mvn/extensions.xml
logstash-2.4.0/vendor/bundle/jruby/1.9/gems/ruby-maven-3.3.12/.mvn/extensions.xml
logstash-2.4.0/Gemfile
logstash-2.4.0/Gemfile.jruby-1.9.lock

将解压后的logstash-2.4.0目录名改成logstash

[root@dev18 srv]# mv logstash-2.4.0 logstash

进入logstash目录，查看logstash目录下有哪些目录和文件~

[root@dev18 srv]# cd logstash
[root@dev18 logstash]# ll
total 160
drwxr-xr-x 2 root root   4096 Mar 17 16:39 bin
-rw-rw-r-- 1 root root 102879 Aug 30  2016 CHANGELOG.md
-rw-rw-r-- 1 root root   2249 Aug 30  2016 CONTRIBUTORS
-rw-rw-r-- 1 root root   4976 Aug 30  2016 Gemfile
-rw-rw-r-- 1 root root  22850 Aug 30  2016 Gemfile.jruby-1.9.lock
drwxr-xr-x 4 root root   4096 Mar 17 16:39 lib
-rw-rw-r-- 1 root root    589 Aug 30  2016 LICENSE
-rw-rw-r-- 1 root root    149 Aug 30  2016 NOTICE.TXT
drwxr-xr-x 4 root root   4096 Mar 17 16:39 vendor
[root@dev18 logstash]#

启动Logstash

进入<Logstash_HOME>/bin目录，

[root@dev18 logstash]# cd bin
[root@dev18 bin]# ll
total 44
-rwxrwxr-x 1 root root 1854 Aug 30  2016 logstash
-rw-rw-r-- 1 root root  689 Aug 30  2016 logstash.bat
-rwxrwxr-x 1 root root 5330 Aug 30  2016 logstash.lib.sh
-rwxrwxr-x 1 root root  439 Aug 30  2016 logstash-plugin
-rw-rw-r-- 1 root root  251 Aug 30  2016 logstash-plugin.bat
-rwxrwxr-x 1 root root  199 Aug 30  2016 plugin
-rw-rw-r-- 1 root root  203 Aug 30  2016 plugin.bat
-rwxrwxr-x 1 root root  322 Aug 30  2016 rspec
-rw-rw-r-- 1 root root  245 Aug 30  2016 rspec.bat
-rw-rw-r-- 1 root root 2947 Aug 30  2016 setup.bat

使用logstash工具即可启动Logstash~~如：

[root@dev18 bin]# ./logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'

示例

Logstash具有较为丰富的输入（input），过滤（filter）以及输出（output）插件。

本文给出两个示例，分别为

标准输入输出
标准输入Redis输出

下面就逐个给出示例~ Here we go~

标准输入输出

在这个示例中，使用最简单的控制台输入（stdin）和控制台输出（stdout），启动命令如下：

[root@dev18 bin]# ./logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'

查看logstash是否正常启动

[root@dev18 ~]# ps -ef|grep logstash
root      1352  1168 14 16:51 pts/0    00:00:39 /usr/java/jdk1.7.0_71/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Xmx1g -Xss2048k -Djffi.boot.library.path=/srv/logstash/vendor/jruby/lib/jni -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/srv/logstash/heapdump.hprof -Xbootclasspath/a:/srv/logstash/vendor/jruby/lib/jruby.jar -classpath :.:/usr/java/jdk1.7.0_71/jre/lib/rt.jar:/usr/java/jdk1.7.0_71/lib/dt.jar:/usr/java/jdk1.7.0_71/lib/tools.jar -Djruby.home=/srv/logstash/vendor/jruby -Djruby.lib=/srv/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /srv/logstash/lib/bootstrap/environment.rb logstash/runner.rb agent -e input{stdin{}}output{stdout{codec=>rubydebug}}
root      1448  1423  0 16:56 pts/2    00:00:00 grep logstash
[root@dev18 ~]#

控制台输入hello logstash，然后看一下输出~~ :）

[root@dev18 bin]# ./logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'
hello logstash
{
       "message" => "hello logstash",
      "@version" => "1",
    "@timestamp" => "2017-03-17T08:53:11.975Z",
          "host" => "dev18.xxxx.xxxx"
}

标准输入Redis输出

在这个示例中，使用控制台输入（stdin），使用Redis订阅作为输出(stdout)~~

因为需要指定Redis的属性，内容较多，所以不像第一个示例那样使用./logstash -e 来启动，本示例采用指定配置文件的方式来启动~

首先，在Logstash安装目录下，创建myconf目录，该目录用于存放配置文件~如：

[root@dev18 logstash]# mkdir myconf
[root@dev18 logstash]# ll
total 164
drwxr-xr-x 2 root root   4096 Mar 17 16:39 bin
-rw-rw-r-- 1 root root 102879 Aug 30  2016 CHANGELOG.md
-rw-rw-r-- 1 root root   2249 Aug 30  2016 CONTRIBUTORS
-rw-rw-r-- 1 root root   4976 Aug 30  2016 Gemfile
-rw-rw-r-- 1 root root  22850 Aug 30  2016 Gemfile.jruby-1.9.lock
drwxr-xr-x 4 root root   4096 Mar 17 16:39 lib
-rw-rw-r-- 1 root root    589 Aug 30  2016 LICENSE
drwxr-xr-x 2 root root   4096 Mar 17 19:20 myconf
-rw-rw-r-- 1 root root    149 Aug 30  2016 NOTICE.TXT
drwxr-xr-x 4 root root   4096 Mar 17 16:39 vendor

然后，创建一个配置文件，名字为 stdin2redis.conf

[root@dev18 logstash]# cd myconf/
[root@dev18 myconf]# vim stdin2redis.conf

stdin2redis.conf文件内容如下：

input {
    stdin { }
}

output {
    # 输出到控制台
    # stdout { }

    # 输出到redis
    redis {
        host => "172.xx.xx.xxx"   # redis主机地址
        port => 6379              # redis端口号
        db => 0                   # redis数据库编号
        data_type => "channel"    # 使用发布/订阅模式
        key => "logstash_channel"  # 发布通道名称
    }
}

指定配置文件，启动Logstash

[root@dev18 bin]# ./logstash -f ../myconf/stdin2redis.conf 
Settings: Default pipeline workers: 2
Pipeline main started

打开Redis客户端，订阅logstash_channel


[root@dev18 src]# ./redis-cli 
127.0.0.1:6379> SUBSCRIBE logstash_channel
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "logstash_channel"
3) (integer) 1

测试，在控制台分别输出三组字符串，分别为“hello logstash”, "hello java" 以及"hello china"

[root@dev18 bin]# ./logstash -f ../myconf/stdin2redis.conf 
Settings: Default pipeline workers: 2
Pipeline main started
hello logstash
hello java
hello china

我们可以看到，Redis客户端显示订阅的内容~

127.0.0.1:6379> SUBSCRIBE logstash_channel
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "logstash_channel"
3) (integer) 1
1) "message"
2) "logstash_channel"
3) "{\"message\":\"hello logstash\",\"@version\":\"1\",\"@timestamp\":\"2017-03-17T11:39:28.884Z\",\"host\":\"dev18.gzhl.zhhl\"}"
1) "message"
2) "logstash_channel"
3) "{\"message\":\"hello java\",\"@version\":\"1\",\"@timestamp\":\"2017-03-17T11:39:49.131Z\",\"host\":\"dev18.gzhl.zhhl\"}"
1) "message"
2) "logstash_channel"
3) "{\"message\":\"hello china\",\"@version\":\"1\",\"@timestamp\":\"2017-03-17T11:39:53.042Z\",\"host\":\"dev18.gzhl.zhhl\"}"

本文就写到这边，鉴于Logstash具有丰富的输入和输出，后续慢慢玩~~

转载于:https://my.oschina.net/wangmengjun/blog/861636