在Linux上flume的配置:
agent.channels = ch-1
agent.sources = src-1
agent.sinks = sink-1
agent.sources.src-1.interceptors = i1
agent.sources.src-1.interceptors.i1.type = timestamp
agent.channels.ch-1.type = memory
agent.channels.ch-1.capacity = 10000000
agent.channels.ch-1.transactionCapacity = 1000
#agent.sinks.sink-1.type = file_roll
#agent.sinks.sink-1.sink.directory=/opt/log/data
#agent.sinks.sink-1.sink.rollInterval = 0
#agent.sinks.sink-1.sink.batchSize=100
#agent.sinks.sink-1.sink.serializer=text
#agent.sinks.sink-1.sink.serializer.appendNewline = false
agent.sinks.sink-1.type = com.baidu.unbiz.flume.sink.RollingByTypeAndDayFileSink
agent.sinks.sink-1.sink.directory=/opt/log/data
agent.sinks.sink-1.channel = ch-1
agent.sinks.sink-1.sink.serializer.appendNewline = false
agent.sources.src-1.type = avro
agent.sources.src-1.bind = 192.168.1.130
agent.sources.src-1.port = 41414
agent.sinks.sink-1.channel = ch-1
agent.sources.src-1.channels= ch-1
程序的log4j.properties的配置:
log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname = 192.168.1.130
log4j.appender.flume.Port = 41414
log4j.appender.flume.UnsafeMode = true
log4j.appender.flume.layout=org.apache.log4j.PatternLayout
log4j.appender.flume.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %p [%c:%L] - %m%n
# configure a class's logger to output to the flume appender
log4j.logger.com.cloudera.flume.flumelog4jtest.LogTestApp = INFO,flume
项目maven的pom.xml 添加配置:
<dependency>
<artifactId>log4j</artifactId>
<groupId>log4j</groupId>
<version>1.2.16</version>
</dependency>
<dependency>
<groupId>org.apache.flume.flume-ng-clients</groupId>
<artifactId>flume-ng-log4jappender</artifactId>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>1.7.2</version>
</dependency>