logback_1-Appenders

一 What’s appender

Logback 把"写日志"这个操作 delegate 给了 Appender. Appender是个接口:

public interface Appender<E> extends LifeCycle, ContextAware, FilterAttachable {

  public String getName();
  public void setName(String name);
  void doAppend(E event); 
}

doAppend() 是带有 泛型 E 的. E 的具体类型和依赖的模块有关. 如果是logback-classic,E 是ILoggingEvent; 如果是logback-access,E是AccessEvent.
Appender.doAppend()负责以恰当的格式将日志输出到磁盘文件 ,socket等.

Appender同时是个 FilterAttachable,一个Appender能够关联一个或多个FilterAttachable.

虽然最终输出LoggingEvent的是Appender,但是 event 的格式化是由 Layout 或者 Encoder干的. 每个 Layout(Encoder) 和 一个 且只跟一个 Appender关联,也就是所谓的 owning appender.

有些 Appender有内置的 或者 固定的 event format.这种情况下, 这些Appender不需要 Layout 或者 Encoder. 比如, SocketAppender的append 就只会序列化,不会 格式化.

二 AppenderBase

AppenderBase 是 Appender的抽象类. AppenderBase 只提供了所有 Appender都必需的基础功能,比如 get/set name , activation status, layout , fliters.
AppenderBase实现了Appender接口的doAppend()方法. doAppend()代码如下(这倒是很核心的代码):

public synchronized void doAppend(E eventObject) {
  // prevent re-entry.
  if (guard) {
    return;
  }
  try {
    guard = true;

    if (!this.started) {
      if (statusRepeatCount++ < ALLOWED_REPEATS) {
        addStatus(new WarnStatus(
            "Attempted to append to non started appender [" + name + "].",this));
      }
      return;
    }
    if (getFilterChainDecision(eventObject) == FilterReply.DENY) {
      return;
    }   
    // ok, we now invoke the derived class's implementation of append
    this.append(eventObject);
  } finally {
    guard = false;
  }
}

doAppend()是synchronized方法, 多个线程可能竞争. 不过 synchronized并不总是很合适. UnsynchronizedAppenderBase (代码如下)是一个没有synchronized版本的实现, 但是两者的实现还是十分类似的. 我们最常用的Appender大多是基于UnsynchronizedAppenderBase的,比如RollingFileAppender, ConsoleAppender等.

 public void doAppend(E eventObject) {
        // WARNING: The guard check MUST be the first statement in the
        // doAppend() method.

        // prevent re-entry.
        if (Boolean.TRUE.equals(guard.get())) {
            return;
        }

        try {
            guard.set(Boolean.TRUE);

            if (!this.started) {
                if (statusRepeatCount++ < ALLOWED_REPEATS) {
                    addStatus(new WarnStatus("Attempted to append to non started appender [" + name + "].", this));
                }
                return;
            }

            if (getFilterChainDecision(eventObject) == FilterReply.DENY) {
                return;
            }

            // ok, we now invoke derived class' implementation of append
            this.append(eventObject);

        } catch (Exception e) {
            if (exceptionCount++ < ALLOWED_REPEATS) {
                addError("Appender [" + name + "] failed to append.", e);
            }
        } finally {
            guard.set(Boolean.FALSE);
        }
    }

三 常用Appender

3.1 OutputStreamAppender

OutputStreamAppender 一般我们不会直接使用它, 它是其他常用 appender 的父类.看类结构就知道:

3.2 ConsoleAppender

本质就是 System.out 或者 System.err(标准输出流), 只不过logback包装后更精细点.
这里给出一个案例:

<configuration>

  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <!-- encoders are assigned the type
         ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
    </encoder>
  </appender>

  <root level="DEBUG">
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

3.3 FileAppender

这个就是最常用的把日志输出到文件了.比如:

<configuration>

  <appender name="FILE" class="ch.qos.logback.core.FileAppender">
    <file>testFile.log</file>
    <append>true</append>
    <!-- set immediateFlush to false for much higher logging throughput -->
    <immediateFlush>true</immediateFlush>
    <!-- encoders are assigned the type
         ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>
        
  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>

3.3.1 Uniquely named files (by timestamp)

我们有的时候希望 应用启动一次就搞一个新的日志文件,可以使用应用启动的时间戳作为日志名字, 怎么玩呢?

<configuration>

  <!-- Insert the current time formatted as "yyyyMMdd'T'HHmmss" under
       the key "bySecond" into the logger context. This value will be
       available to all subsequent configuration elements. -->
  <timestamp key="bySecond" datePattern="yyyyMMdd'T'HHmmss"/>

  <appender name="FILE" class="ch.qos.logback.core.FileAppender">
    <!-- use the previously created timestamp to create a uniquely
         named log file -->
    <file>log-${bySecond}.txt</file>
    <encoder>
      <pattern>%logger{35} - %msg%n</pattern>
    </encoder>
  </appender>

  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>

3.4 RollingFileAppender

跟FileAppender相比,RollingFileAppender有了 roll 文件的功能. 比如, RollingFileAppender可以在一开始输出日志到 log.txt,但是当满足一些条件时,将log.txt 给改名为log.backup.txt.

RollingFileAppender有二个核心组件, RollingPolicy和TriggeringPolicy.

RollingPolicy 用来做 rollover的动作, 解决了 do what 的问题;
TriggeringPolicy用来决定 rollover何时发生,解决了 when 的问题.

要想使用RollingFileAppender, 两个 Policy都要配置, 不过在实践层面上一个类可同时实现RollingPolicy TriggeringPolicy.

3.4.1 Overview of rolling policies

RollingPolicy负责 rollover , 这包括 file moving and renaming.

看这个代码:

public interface RollingPolicy extends LifeCycle {

   // The rollover method accomplishes the work involved in archiving the current log file.  
  public void rollover() throws RolloverFailure;
  // The getActiveFileName() method is called to compute the file 
  //name of the current log file (where live logs are written to).
  public String getActiveFileName();
  //As indicated by getCompressionMode method a RollingPolicy is also responsible for determining the compression mode.
  public CompressionMode getCompressionMode();
  // Lastly, a RollingPolicy is given a reference to its parent via the setParent method.
  public void setParent(FileAppender appender);
}

3.4.2 TimeBasedRollingPolicy

TimeBasedRollingPolicy 根据时间,比如日, 月来 roll日志. TimeBasedRollingPolicy 同时实现了 RollingPolicy和TriggeringPolicy.
举个栗子:

<configuration>
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>logFile.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <!-- daily rollover -->
      <fileNamePattern>logFile.%d{yyyy-MM-dd}.log</fileNamePattern>

      <!-- keep 30 days' worth of history capped at 3GB total size -->
      <maxHistory>30</maxHistory>
      <totalSizeCap>3GB</totalSizeCap>

    </rollingPolicy>

    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender> 

  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>

3.4.3 Size and time based rolling policy

如果你既想根据 日期 归档, 同时想限制每个log file 大小,可以用 SizeAndTimeBasedRollingPolicy.
不过其实 前面说的 TimeBasedRollingPolicy 已经允许限制 归档日志总大小了.假如你只想限制一下归档日志总大小, 把TimeBasedRollingPolicy的 totalSizeCap属性调到合理值即可.

<configuration>
  <appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>mylog.txt</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
      <!-- rollover daily -->
      <fileNamePattern>mylog-%d{yyyy-MM-dd}.%i.txt</fileNamePattern>
       <!-- each file should be at most 100MB, keep 60 days worth of history, but at most 20GB -->
       <maxFileSize>100MB</maxFileSize>    
       <maxHistory>60</maxHistory>
       <totalSizeCap>20GB</totalSizeCap>
    </rollingPolicy>
    <encoder>
      <pattern>%msg%n</pattern>
    </encoder>
  </appender>


  <root level="DEBUG">
    <appender-ref ref="ROLLING" />
  </root>

</configuration>

3.4.4 FixedWindowRollingPolicy

FixedWindowRollingPolicy根据 固定时间窗口重命名文件名.
fileNamePattern属性代表 归档日志文件 的文件名. 这个属性是必须的且要包括 %i 的整数token.

<configuration>
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>test.log</file>

    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>tests.%i.log.zip</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>3</maxIndex>
    </rollingPolicy>

    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>
        
  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>

3.5 Overview of triggering policies

TriggeringPolicy就是确定RollingFileAppender何时 rollover的.TriggeringPolicy定义:

public interface TriggeringPolicy<E> extends LifeCycle {

  public boolean isTriggeringEvent(final File activeFile, final <E> event);
}

3.5.1 SizeBasedTriggeringPolicy

SizeBasedTriggeringPolicy会检查当前 active file的大小. 如果它已经大过指定值了, 就会触发RollingFileAppender对当前 active file的 rollover

SizeBasedTriggeringPolicy只接受一个参数,也即是maxFileSize,默认 10 MB.

这里使用 RollingFileAppender ,结合 SizeBasedTriggeringPolicy给了个例子:

<configuration>
  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>test.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <fileNamePattern>test.%i.log.zip</fileNamePattern>
      <minIndex>1</minIndex>
      <maxIndex>3</maxIndex>
    </rollingPolicy>

    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <maxFileSize>5MB</maxFileSize>
    </triggeringPolicy>
    <encoder>
      <pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
    </encoder>
  </appender>
        
  <root level="DEBUG">
    <appender-ref ref="FILE" />
  </root>
</configuration>

3.6 Logback Classic

Logback Classic基于 Logback Core了.
Logback Classic 提供了更多的 appender ,适用于不同场景, 比如:

• SocketAppender and SSLSocketAppender
• ServerSocketAppender and SSLServerSocketAppender
• SMTPAppender
SMTPAppender有个有意思的功能, 当你的日志输出了ERROR 或者WARN级别日志时,可以把错误日志通过邮件发给你.
  同时, 甚至可以通过 实现 Evaluator 接口来自定义具体触发条件.如果你们公司没有异常检测告警系统, 可以使用这种方式来自定义告警,还是很实用的.
• DBAppender
  这个更有意思了. 我们习惯于将auditlog 打印在日志里, 不过对于中小应用,是不是将日志持久化到数据库也可以呢?
• SyslogAppender
• SiftingAppender
  这个简直亮爆了.SiftingAppender能够用来 根据某个给定运行值属性差异化打日志. 比如, 根据 user sessions区分日志,得到 一个user 一个log file .
• AsyncAppender
  这种appender 更是不得不提了. AsyncAppender允许异步输出日志.AsyncAppender更像是一个 dispatcher, 它必须依赖其它的 appender.

AsyncAppender有几个点需要重点注意:

• LOSSY BY DEFAULT IF 80% FULL. AsyncAppender 内部有一个BlockingQueue. AsyncAppender的一个worker 线程不断从 Q 里取出 loggingEvent ,并 dispatch 给其它 appender . 但默认情况下, AsyncAppender会在 Q 的大小超过 80% 时就丢掉 TRACE, DEBUG and INFO 级别的日志. 这其实是为了保障性能. 而且更为关键的 ERROR 日志并没有丢.
• APPLICATION STOP/REDEPLOY. 应用关闭或者重新部署的时候, AsyncAppender需要停下来关闭和回收 worker线程, 并把Q中的loggingEvent flush 出去. 这可以通过 关闭LoggerContxt实现, 关闭LoggerContxt会关闭所有appender,包括 AsyncAppender实例. AsyncAppender将会等待 worker 线程flush ,但最多等待 maxFlushTime. 假如发现在关闭 LoggerContxt时有LoggingEvent 丢失了,那么就要考虑增大 maxFlushTime了
• POST SHUTDOWN CLEANUP. JVM退出的方式不同, worker thread可能被 interrupt 导致 Q 中LoggingEvent 不能及时flush 出去. 这通常 是由于 LoggerContxt没有关闭干净, 或者JVM关闭姿势不正确. 为了避免worker thread被中断掉, 可以在 JVM shutdown hook 中关闭 LoggerContxt.
  A shutdown hook may also be the preferred method for cleanly shutting down Logback when other shutdown hooks attempt to log events.
• 最最关键的是 neverBlock = true (default false) 这个配置一定要记得加上,否则有可能logger.info() 阻塞住,导致业务线程逻辑等很久.
  举个栗子:

<configuration>
 <appender name="FILE" class="ch.qos.logback.core.FileAppender">
   <file>myapp.log</file>
   <encoder>
     <pattern>%logger{35} - %msg%n</pattern>
   </encoder>
 </appender>

 <appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
   <appender-ref ref="FILE" />
 </appender>

 <root level="DEBUG">
   <appender-ref ref="ASYNC" />
 </root>
</configuration>

3.7 Customer Appender

用户也可以自己写一个 Appender.
不过说实话,极少有此场景

3.8 Logback Access

logback-classic中的大部分 appender在 Logback Access也找到类似的.
常用的(也不算常用吧)的appender有下面这些

• SocketAppender and SSLSocketAppender
• ServerSocketAppender and SSLServerSocketAppender
• SMTPAppender
• DBAppender
• SiftingAppender

一些中间件,比如tomcat ,nginx 会有记录 access log 的需求.