【Flink系列】配置管理rockmq-flink产生的rocketmq-client日志

Flink任务集成了rockmq-flink用于订阅消费rocketmq的消息，在任务运行过程中发现会在系统的${username}/logs/rocketmqlogs目录下产生rocketmq_client.log日志，并且这个日志累积和滚动的很快，考虑将其配置和控制起来。

参考rocketmq-client-4.7.1的ClientLogger源码

 private static synchronized Appender createClientAppender() {
        String clientLogRoot = System.getProperty("rocketmq.client.logRoot", System.getProperty("user.home") + "/logs/rocketmqlogs");
        String clientLogMaxIndex = System.getProperty("rocketmq.client.logFileMaxIndex", "10");
        String clientLogFileName = System.getProperty("rocketmq.client.logFileName", "rocketmq_client.log");
        String maxFileSize = System.getProperty("rocketmq.client.logFileMaxSize", "1073741824");
        String asyncQueueSize = System.getProperty("rocketmq.client.logAsyncQueueSize", "1024");
        String logFileName = clientLogRoot + "/" + clientLogFileName;
        int maxFileIndex = Integer.parseInt(clientLogMaxIndex);
        int queueSize = Integer.parseInt(asyncQueueSize);
        Layout layout = LoggingBuilder.newLayoutBuilder().withDefaultLayout().build();
        Appender rocketmqClientAppender = LoggingBuilder.newAppenderBuilder().withRollingFileAppender(logFileName, maxFileSize, maxFileIndex).withAsync(false, queueSize).withName("RocketmqClientAppender").withLayout(layout).build();
        Logger.getRootLogger().addAppender(rocketmqClientAppender);
        return rocketmqClientAppender;
    }

    private static InternalLogger createLogger(String loggerName) {
        String clientLogLevel = System.getProperty("rocketmq.client.logLevel", "INFO");
        boolean additive = "true".equalsIgnoreCase(System.getProperty("rocketmq.client.log.additive"));
        InternalLogger logger = InternalLoggerFactory.getLogger(loggerName);
        InnerLogger innerLogger = (InnerLogger)logger;
        Logger realLogger = innerLogger.getLogger();
        realLogger.addAppender(new ClientLogger.AppenderProxy());
        realLogger.setLevel(Level.toLevel(clientLogLevel));
        realLogger.setAdditivity(additive);
        return logger;
    }

    public static InternalLogger getLog() {
        return CLIENT_LOGGER;
    }

    static {
        if (!CLIENT_USE_SLF4J) {
            InternalLoggerFactory.setCurrentLoggerType("inner");
            CLIENT_LOGGER = createLogger("RocketmqClient");
            createLogger("RocketmqCommon");
            createLogger("RocketmqRemoting");
        } else {
            CLIENT_LOGGER = InternalLoggerFactory.getLogger("RocketmqClient");
        }

    }

如果在Flink任务启动时，没有找到

rocketmq.client.logUseSlf4j

这个系统变量， rocketmq-client就会按照自己的方式去打日志了，就是前面提到的，日志都放到${username}/logs/rocketmqclient目录下面。

所以我们需要在Flink启动时加上这一系统变量，打开flink的conf文件，加上以下配置：

env.java.opts: "-Drocketmq.client.logUseSlf4j=true"

同时配置flink的logback.xml文件，添加如下配置：

    <appender name="RocketmqClientAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>rocketmq_client.log</file>
        <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
          <fileNamePattern>rocketmq_client.%i</fileNamePattern>
          <minIndex>1</minIndex>
          <maxIndex>10</maxIndex>
        </rollingPolicy>
        <append>false</append>
        <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
          <maxFileSize>10MB</maxFileSize>
        </triggeringPolicy>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{60} %X{sourceThread} - %msg%n</pattern>
        </encoder>
    </appender>
    <logger name="RocketmqClient" level="ERROR">
        <appender-ref ref="RocketmqClientAppender"/>
    </logger>

也可以根据自己的需求修改配置。

配置好以后，重新启动任务，会发现日志已经不写到原先的${username}/logs/rocketmqclient目录中去了。