docker 安装filebeat7.8.0

环境说明

操作系统：CentOS 7.7

docker版本：20.10.7

ip地址：192.168.38.129

logstash: 7.8.0

安装

• 拉取镜像

docker pull elastic/filebeat:7.8.0

• 创建卷目录

mkdir -p /docker/elk/filebeat
vi /docker/elk/filebeat/filebeat.yml

• 配置如下（匹配合并行，匹配默认时间类型开始行 2020-08-05 16:21:51）

filebeat.inputs:
# 类型
- type: log
  # 开启
  enabled: true
  # 日志文件路径
  paths:
    - /var/log/app/*.log
  multiline:
    # 匹配时间格式： 2020-08-05 16:21:51
    pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}'
    # pattern: '^\['
    negate: true
    match: after
  fields: 
    log_from: server1
#============================= Filebeat modules ===============================
filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false
#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["192.168.38.129:5044"]

• 创建容器并启动

docker run -d --name filebeat \
-v /docker/elk/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml \
-v /var/log/app:/var/log/app \
elastic/filebeat:7.8.0

• 添加测试数据
  vi /var/log/app/spring.log

 java.lang.IllegalStateException: java.lang.IllegalStateException: Logback configuration error detected:
ERROR in ch.qos.logback.core.joran.action.AppenderRefAction - Could not find an appender named [ECS_JSON_FILE]. Did you define it below instead of above in the configuration file?
ERROR in ch.qos.logback.core.joran.action.AppenderRefAction - See http://logback.qos.ch/codes.html#appender_order for more details.
        at org.springframework.boot.context.logging.LoggingApplicationListener.initializeSystem(LoggingApplicationListener.java:328)
        at org.springframework.boot.context.logging.LoggingApplicationListener.initialize(LoggingApplicationListener.java:282)
        at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEnvironmentPreparedEvent(LoggingApplicationListener.java:240)
        at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.java:216)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131)
        at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:82)
        at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:63)
        at java.util.ArrayList.forEach(ArrayList.java:1257)
        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:117)
        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:111)
        at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:62)
        at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:374)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:332)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332)
        at com.filebeat.demo.EsFilebeatApplication.main(EsFilebeatApplication.java:10)
Caused by: java.lang.IllegalStateException: Logback configuration error detected:
ERROR in ch.qos.logback.core.joran.action.AppenderRefAction - Could not find an appender named [ECS_JSON_FILE]. Did you define it below instead of above in the configuration file?
ERROR in ch.qos.logback.core.joran.action.AppenderRefAction - See http://logback.qos.ch/codes.html#appender_order for more details.
        at org.springframework.boot.logging.logback.LogbackLoggingSystem.loadConfiguration(LogbackLoggingSystem.java:179)
        at org.springframework.boot.logging.AbstractLoggingSystem.initializeWithConventions(AbstractLoggingSystem.java:80)
        at org.springframework.boot.logging.AbstractLoggingSystem.initialize(AbstractLoggingSystem.java:60)
        at org.springframework.boot.logging.logback.LogbackLoggingSystem.initialize(LogbackLoggingSystem.java:132)
        at org.springframework.boot.context.logging.LoggingApplicationListener.initializeSystem(LoggingApplicationListener.java:313)
        ... 18 common frames omitted

2021-07-22 22:32:47.757  INFO 24348 --- [           main] com.kl.sso.client.SsoClientApplication   : No active profile set, falling back to default profiles: default
2021-07-22 22:32:49.788  INFO 24348 --- [           main] o.s.cloud.context.scope.GenericScope     : BeanFactory id=db347cb3-6006-369d-818c-91b35530d013
2021-07-22 22:32:49.841  INFO 24348 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration$$EnhancerBySpringCGLIB$$118ca32a] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-07-22 22:32:50.359  INFO 24348 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8081 (http)
2021-07-22 22:32:50.380  INFO 24348 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-07-22 22:32:50.380  INFO 24348 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.39]
2021-07-22 22:32:50.933  INFO 24348 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-07-22 22:32:50.933  INFO 24348 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 3116 ms
2021-07-22 22:32:51.571  WARN 24348 --- [           main] o.s.s.o.p.t.s.JwtAccessTokenConverter    : Unable to create an RSA verifier from verifierKey (ignoreable if using MAC)
2021-07-22 22:32:51.811  INFO 24348 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2021-07-22 22:32:52.807  INFO 24348 --- [           main] .s.s.UserDetailsServiceAutoConfiguration :

Using generated security password: a0f33e60-df32-4361-b4da-2d18039df29b

2021-07-22 22:32:53.059  INFO 24348 --- [           main] o.s.s.web.DefaultSecurityFilterChain     : Creating filter chain: Ant [pattern='/**'], [org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@33ecbd6c, org.springframework.security.web.context.SecurityContextPersistenceFilter@2373ad99, org.springframework.security.web.header.HeaderWriterFilter@5793b87, org.springframework.security.web.csrf.CsrfFilter@1c8f6a90, org.springframework.security.web.authentication.logout.LogoutFilter@4c2af006, org.springframework.security.oauth2.client.filter.OAuth2ClientAuthenticationProcessingFilter@5c723f2d, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@4993febc, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@7a360554, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@432f521f, org.springframework.security.web.session.SessionManagementFilter@512575e9, org.springframework.security.web.access.ExceptionTranslationFilter@352ed70d, org.springframework.security.web.access.intercept.FilterSecurityInterceptor@707b1a44]
2021-07-22 22:32:54.706  INFO 24348 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 2 endpoint(s) beneath base path '/actuator'
2021-07-22 22:32:54.951  INFO 24348 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8081 (http) with context path ''
2021-07-22 22:32:54.990  INFO 24348 --- [           main] com.kl.sso.client.SsoClientApplication   : Started SsoClientApplication in 11.807 seconds (JVM running for 15.224)
2021-07-22 22:32:57.051  INFO 24348 --- [3)-192.168.38.1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2021-07-22 22:32:57.055  INFO 24348 --- [3)-192.168.38.1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2021-07-22 22:32:57.069  INFO 24348 --- [3)-192.168.38.1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 14 ms

• 在kibana，查看es中数据