Common Log Formats and How To Parse Them

This page is a glossary of common log formats that can be parsed with the Tail input plugin.

  • Apache Access Log

    Use format apache2 as shown below:

      <source>
          @type tail
          format apache2
          tag apache.access
          path /var/log/apache2/access.log
      </source>
    
  • Apache Error Log

    Use a regular expression. See the format field in the following sample configuration.

      <source>
          @type tail
          format /^\[[^ ]* (?<time>[^\]]*)\] \[(?<level>[^\]]*)\] \[pid (?<pid>[^\]]*)\] \[client (?<client>[^\]]*)\] (?<message>.*)$/
          tag apache.error
          path /var/log/apache2/error.log
      </source>
    

    Depending on your particular error log format, you may need to adjust the regular expression above. You can test your format using fluentd-ui’s in_tail editor or Fluentular.

  • Maillog

    Use a regular expression. See the format field in the following sample configuration.

      <source>
          @type tail
          format /^(?<time>[^ ]+) (?<host>[^ ]+) (?<process>[^:]+): (?<message>((?<key>[^ :]+)[ :])? ?((to|from)=<(?<address>[^>]+)>)?.*)$/
          tag postfix.maillog
          path /var/log/maillog
      </source>
    
  • Nginx Access Log

    Use format nginx as shown below:

      <source>
          @type tail
          format nginx
          tag nginx.access
          path /var/log/nginx/access.log
      </source>
    
  • Nginx Error Log

    Use the format* and multiline_flush_interval fields in the following sample configuration. Applications running under Nginx can output multi-line errors including stack traces, so the multiline mode is a good fit.

      <source>
          @type tail
          tag nginx.error
          path /var/log/nginx/error.log
    
          format multiline
          format_firstline /^\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2} \[\w+\] (?<pid>\d+).(?<tid>\d+): /
          format1 /^(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?<log_level>\w+)\] (?<pid>\d+).(?<tid>\d+): (?<message>.*)/
          multiline_flush_interval 3s
      </source>
    

    If you know your error log will only contain single lines, you can use the below simpler configuration with just a format.

      <source>
          @type tail
          format /^(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?<log_level>\w+)\] (?<pid>\d+).(?<tid>\d+): (?<message>.*)$/
          tag nginx.error
          path /var/log/nginx/error.log
      </source>
    
  • GlusterFS Logs

    Use the GlusterFS input plugin.

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值