flume与log4j的整合

1.导入依赖包jar

 1 <dependencies>
 2 <dependency>            
 3 <groupId>org.apache.flume.flume-ng-clients</groupId>            
 4 <artifactId>flume-ng-log4jappender</artifactId>            <version>1.6.0</version>        
 5 </dependency>
 6 
 7 
 8 <!-- log4j 依赖包-->        
 9 <dependency>            
10 <groupId>log4j</groupId>            
11 <artifactId>log4j</artifactId>            
12 <version>1.2.17</version>        
13 </dependency>    
14     
15 <dependency>            
16 <groupId>org.slf4j</groupId>            
17 <artifactId>slf4j-api</artifactId>            
18 <version>1.7.5</version>        
19 </dependency>    
20     
21 <dependency>            
22 <groupId>org.slf4j</groupId>            
23 <artifactId>slf4j-log4j12</artifactId>            
24 <version>1.7.5</version>        
25 </dependency>
26 </dependencies>

 

2.配置文件log4j.properties

 1 #
 2 # Licensed to the Apache Software Foundation (ASF) under one
 3 # or more contributor license agreements.  See the NOTICE file
 4 # distributed with this work for additional information
 5 # regarding copyright ownership.  The ASF licenses this file
 6 # to you under the Apache License, Version 2.0 (the
 7 # "License"); you may not use this file except in compliance
 8 # with the License.  You may obtain a copy of the License at
 9 #
10 #  http://www.apache.org/licenses/LICENSE-2.0
11 #
12 # Unless required by applicable law or agreed to in writing,
13 # software distributed under the License is distributed on an
14 # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15 # KIND, either express or implied.  See the License for the
16 # specific language governing permissions and limitations
17 # under the License.
18 #
19 
20 # Define some default values that can be overridden by system properties.
21 #
22 # For testing, it may also be convenient to specify
23 # -Dflume.root.logger=DEBUG,console when launching flume.
24 
25 #flume.root.logger=DEBUG,console
26 flume.root.logger=INFO,DAILY
27 flume.log.dir=./logs
28 flume.log.file=flume.log
29 
30 
31 # Define the root logger to the system property "flume.root.logger".
32 log4j.rootLogger=${flume.root.logger}
33 
34 
35 # Stock log4j rolling file appender
36 # Default log rotation configuration
37 #按文件大小回滚RollingFileAppender
38 log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender
39 #当产生日志文件大于MaxFileSize，进行回滚
40 log4j.appender.LOGFILE.MaxFileSize=20MB
41 #最大回滚数
42 log4j.appender.LOGFILE.MaxBackupIndex=30
43 #产生日志文件的路径及文件名
44 log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file}
45 #自定义格式
46 log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout
47 log4j.appender.LOGFILE.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n
48 
49 
50 # Warning: If you enable the following appender it will fill up your disk if you don't have a cleanup job!
51 # This uses the updated rolling file appender from log4j-extras that supports a reliable time-based rolling policy.
52 # See http://logging.apache.org/log4j/companions/extras/apidocs/org/apache/log4j/rolling/TimeBasedRollingPolicy.html
53 
54 # console
55 # Add "console" to flume.root.logger above if you want to use this
56 log4j.appender.console=org.apache.log4j.ConsoleAppender
57 log4j.appender.console.target=System.err
58 log4j.appender.console.layout=org.apache.log4j.PatternLayout
59 log4j.appender.console.layout.ConversionPattern=%d (%t) [%p - %l] %m%n

把上面的log4j配置文件放在flume的conf目录下

3. 启动flume

　　使用命令：./bin/flume-ng agent --conf conf --conf-file conf/flume2CustomSink.conf --name a1

     注意：不要在命令后面加-Dflume.root.logger=INFO,console    即不要用这命令：./bin/flume-ng agent --conf conf --conf-file conf/flume2CustomSink.conf --name a1 -Dflume.root.logger=INFO,console

　因为使用这个命令就修改了flume.root.logger的值，变成打印在控制台上了。这和我们要打印成日志文件的目的相违背。除非你想要在控制台看打印信息。

 

4.运行成功后可以看到在flume根目录下生成了  ./logs/flume.log 文件

     关于flume的安装和部署 请参照：https://www.cnblogs.com/hoboo/p/9779104.html

 

最后附上log4jPatternLayout输出格式：

ConversionPattern属性：

%m 输出代码中指定的消息；

%M 输出打印该条日志的方法名；

%p 输出优先级，即DEBUG，INFO，WARN，ERROR，FATAL；

%r 输出自应用启动到输出该log信息耗费的毫秒数；

%c 输出所属的类目，通常就是所在类的全名；

%t 输出产生该日志事件的线程名；

%n 输出一个回车换行符，Windows平台为"rn”，Unix平台为"n”；

%d 输出日志时间点的日期或时间，默认格式为ISO8601，也可以在其后指定格式，比如：%d{yyyy-MM-dd HH:mm:ss,SSS}，输出类似：2002-10-18 22:10:28,921；

%l 输出日志事件的发生位置，及在代码中的行数；

转载于:https://www.cnblogs.com/hoboo/p/9890547.html