ELK集群部署(四)之部署Logstash

部署LogStash数据处理器

需要java
yum install -y java-1.8.0-openjdk.x86_64

cd /usr/local/src
wget https://artifacts.elastic.co/downloads/logstash/logstash-7.7.1.tar.gz
tar -zxf logstash-7.7.1.tar.gz

这个配置是获取filebeat的数据，输出到ES
================================================================================
cat > /usr/local/src/logstash-7.7.1/config/fb_oslog.yml <<EOF
input {
    beats {
        port => 5044
    }
}
output {
  elasticsearch {
    hosts => ["192.168.56.101:9200","192.168.56.102:9200","192.168.56.103:9200"]  # 定义es服务器的ip
    index => "syslog-%{+YYYY.MM}" # 定义索引
  }
}
EOF

===========================================从kafka接收数据，输出到es=====================================
从名为kafka的topic获取数据，输出到ES集群，ES端索引名为kafka-2022.07
cat > /usr/local/src/logstash-7.7.1/config/kafka.yml <<EOF
input{
 kafka{
   bootstrap_servers=> ["192.168.56.103:9092"]
   topics=> ["kafka"]
 }
}

output{
 # 调试的时候可以打开查看控制台信息
 #stdout {
  #codec=>rubydebug
 #}
 elasticsearch {
  hosts=> ["192.168.56.101:9200","192.168.56.102:9200","192.168.56.103:9200"]
  index=> "kafka-%{+YYYY.MM}"
 }
}
EOF

启动logstash

/usr/local/src/logstash-7.7.1/bin/logstash -f /usr/local/src/logstash-7.7.1/config/fb_oslog.yml

后台运行
nohup /usr/local/src/logstash-7.7.1/bin/logstash -f /usr/local/src/logstash-7.7.1/config/fb_oslog.yml &

如果提示could not find java; set JAVA_HOME or ensure java is in PATH

yum install -y java-1.8.0-openjdk.x86_64