Flink任务集成了rockmq-flink用于订阅消费rocketmq的消息,在任务运行过程中发现会在系统的${username}/logs/rocketmqlogs目录下产生rocketmq_client.log日志,并且这个日志累积和滚动的很快,考虑将其配置和控制起来。
参考rocketmq-client-4.7.1的ClientLogger源码
private static synchronized Appender createClientAppender() {
String clientLogRoot = System.getProperty("rocketmq.client.logRoot", System.getProperty("user.home") + "/logs/rocketmqlogs");
String clientLogMaxIndex = System.getProperty("rocketmq.client.logFileMaxIndex", "10");
String clientLogFileName = System.getProperty("rocketmq.client.logFileName", "rocketmq_client.log");
String maxFileSize = System.getProperty("rocketmq.client.logFileMaxSize", "1073741824");
String asyncQueueSize = System.getProperty("rocketmq.client.logAsyncQueueSize", "1024");
String logFileName = clientLogRoot + "/" + clientLogFileName;
int maxFileIndex = Integer.parseInt(clientLogMaxIndex);
int queueSize = Integer.parseInt(asyncQueueSize);
Layout layout = LoggingBuilder.newLayoutBuilder().withDefaultLayout().build();
Appender rocketmqClientAppender = LoggingBuilder.newAppenderBuilder().withRollingFileAppender(logFileName, maxFileSize, maxFileIndex).withAsync(false, queueSize).withName("RocketmqClientAppender").withLayout(layout).build();
Logger.getRootLogger().addAppender(rocketmqClientAppender);
return rocketmqClientAppender;
}
private static InternalLogger createLogger(String loggerName) {
String clientLogLevel = System.getProperty("rocketmq.client.logLevel", "INFO");
boolean additive = "true".equalsIgnoreCase(System.getProperty("rocketmq.client.log.additive"));
InternalLogger logger = InternalLoggerFactory.getLogger(loggerName);
InnerLogger innerLogger = (InnerLogger)logger;
Logger realLogger = innerLogger.getLogger();
realLogger.addAppender(new ClientLogger.AppenderProxy());
realLogger.setLevel(Level.toLevel(clientLogLevel));
realLogger.setAdditivity(additive);
return logger;
}
public static InternalLogger getLog() {
return CLIENT_LOGGER;
}
static {
if (!CLIENT_USE_SLF4J) {
InternalLoggerFactory.setCurrentLoggerType("inner");
CLIENT_LOGGER = createLogger("RocketmqClient");
createLogger("RocketmqCommon");
createLogger("RocketmqRemoting");
} else {
CLIENT_LOGGER = InternalLoggerFactory.getLogger("RocketmqClient");
}
}
如果在Flink任务启动时,没有找到
rocketmq.client.logUseSlf4j
这个系统变量, rocketmq-client就会按照自己的方式去打日志了,就是前面提到的,日志都放到${username}/logs/rocketmqclient目录下面。
所以我们需要在Flink启动时加上这一系统变量,打开flink的conf文件,加上以下配置:
env.java.opts: "-Drocketmq.client.logUseSlf4j=true"
同时配置flink的logback.xml文件,添加如下配置:
<appender name="RocketmqClientAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>rocketmq_client.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>rocketmq_client.%i</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>10</maxIndex>
</rollingPolicy>
<append>false</append>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>10MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{60} %X{sourceThread} - %msg%n</pattern>
</encoder>
</appender>
<logger name="RocketmqClient" level="ERROR">
<appender-ref ref="RocketmqClientAppender"/>
</logger>
也可以根据自己的需求修改配置。
配置好以后,重新启动任务,会发现日志已经不写到原先的${username}/logs/rocketmqclient目录中去了。