logstash 笔记

only focus on use

abstraction layer separate the representation

logstash版本基于7.10

https://www.elastic.co/guide/en/logstash/7.10/index.html

1. 测试logstach

cd logstash-7.10.2
bin/logstash -e 'input { stdin { } } output { stdout {} }'

2. skeleton

https://www.elastic.co/guide/en/logstash/7.10/advanced-pipeline.html

input {
    beats {
        port => "5044"
    }
}
# The filter part of this file is commented out to indicate that it is
# optional.
# filter {
#
# }
output {
    stdout { codec => rubydebug }
}

The --config.test_and_exit option parses your configuration file and reports any errors
The --config.reload.automatic option enables automatic config reloading so that you don’t have to stop and restart Logstash every time you modify the configuration file.

3. plugins

The grok filter plugin enables you to parse the unstructured log data into something structured and queryable.

debug grok expression
https://www.elastic.co/guide/en/kibana/7.10/xpack-grokdebugger.html


https://www.elastic.co/guide/en/elasticsearch/reference/7.10/grok-processor.html#grok-basics

pattern的语法：
The syntax for reusing a grok pattern comes in three forms: %{SYNTAX:SEMANTIC}, %{SYNTAX}, %{SYNTAX:SEMANTIC:TYPE}