Linux 安装logstash

1、下载logstash(地址：https://www.elastic.co/downloads/logstash)

wget https://artifacts.elastic.co/downloads/logstash/logstash-7.2.0.tar.gz

tar zxvf logstash-7.2.0.tar.gz

mv logstash-7.2.0.tar.gz logstash

2、运行logstash

报错：内存不足

 

解决方法：

# vim config/jvm.options

将-Xms1g改为 -Xms512m

或 修改系统变量

echo 1 > /proc/sys/vm/overcommit_memory

3、设置logstash.conf配置

# cp logstash-sample.conf logstash.conf

# vim logstash.conf

4、Logstash 读取kafka内容

# Sample Logstash configuration for creating a simple

# Beats -> Logstash -> Elasticsearch pipeline.

 

input {

  kafka {

    bootstrap_servers => "localhost:9092"

    topics => ["test_rhj"]

    group_id => "test_rhj"

    consumer_threads => 1

    decorate_events => true

    codec => json{

      charset => "UTF-8"

    }

  }

}

 

filter {

  json{

    source => "message"

    target => "message"

  }

}

 

output {

# 将kafka的内容写入到文件

  elasticsearch {

    hosts => "http://192.168.10.6:9200"

    index => "aa"

    #user => "elastic"

    #password => "changeme"

  }

#  将kafka的内容写入到文件

#  file{

#    path => "/home/test.log"

#    flush_interval => 0

#  }

}

执行操作

# bin/logstash -f config/logstash.conf