这里是引用
logstash采集Java日志文本文件配合grok收集到elasticsearch简单示例
logstash 配置
input {
file {
path => "C:/Users/Administrator/Desktop/info*.log"
type => "log-info"
start_position => "beginning"
# 多行文本换行处理 (日志文件以日期时间格式开头,所以按日期处理换行)
codec => multiline {
pattern => "^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}\s"
negate => true
what => "previous"
}}
}
filter {
grok {
# 使用正则将日志拆分成多列,方便查询
# 原日志文件使用log4j生成,格式为 : %d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %class{36} %L %M - %msg%xEx%n
match => { "message" => "(?<datetime>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}) %{NOTSPACE:level} (?<className> (?:[a-zA-Z$_][a-zA-Z$_0-9]*\.)*[a-zA-Z$_][a-zA-Z$_0-9]*) %{NUMBER:line} %{NOTSPACE:method} - %{GREEDYDATA:msg}" }
overwrite => ["message"]
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "log4j-info-%{+YYYY.MM.dd}"
codec => "json"
}
}
日志文件示例
2020-08-10 14:47:56.204 INFO org.springframework.boot.StartupInfoLogger 50 logStarting - Starting xxxxx on pc-111with PID 14520 (started by Administrator in E:\worker\XXXX\XXX\XXXX)
2020-08-10 14:47:56.214 INFO org.springframework.boot.SpringApplication 679 logStartupProfileInfo - The following profiles are active: dev
2020-08-10 14:47:57.723 INFO org.springframework.data.repository.config.RepositoryConfigurationDelegate 244 multipleStoresDetected - Multiple Spring Data modules found, entering strict repository configuration mode!
2020-08-10 14:47:57.733 INFO org.springframework.data.repository.config.RepositoryConfigurationDelegate 126 registerRepositoriesIn - Bootstrapping Spring Data repositories in DEFAULT mode.
2020-08-10 14:47:57.803 INFO org.springframework.data.repository.config.RepositoryConfigurationDelegate 182 registerRepositoriesIn - Finished Spring Data repository scanning in 50ms. Found 0 repository interfaces.
2020-08-10 14:47:57.813 INFO org.springframework.data.repository.config.RepositoryConfigurationDelegate 244 multipleStoresDetected - Multiple Spring Data modules found, entering strict repository configuration mode!
2020-08-10 14:47:57.813 INFO org.springframework.data.repository.config.RepositoryConfigurationDelegate 126 registerRepositoriesIn - Bootstrapping Spring Data repositories in DEFAULT mode.
2020-08-10 14:47:57.853 INFO org.springframework.data.repository.config.RepositoryConfigurationDelegate 182 registerRepositoriesIn - Finished Spring Data repository scanning in 30ms. Found 0 repository interfaces.
2020-08-10 14:47:58.602 INFO org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker 330 postProcessAfterInitialization - Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransac