apache flume 配置存储在Linux本地服务器

在Linux上flume的配置：

agent.channels = ch-1
agent.sources = src-1
agent.sinks = sink-1

agent.sources.src-1.interceptors = i1
agent.sources.src-1.interceptors.i1.type = timestamp

agent.channels.ch-1.type = memory
agent.channels.ch-1.capacity = 10000000
agent.channels.ch-1.transactionCapacity = 1000

#agent.sinks.sink-1.type = file_roll
#agent.sinks.sink-1.sink.directory=/opt/log/data
#agent.sinks.sink-1.sink.rollInterval = 0
#agent.sinks.sink-1.sink.batchSize=100
#agent.sinks.sink-1.sink.serializer=text
#agent.sinks.sink-1.sink.serializer.appendNewline = false

agent.sinks.sink-1.type = com.baidu.unbiz.flume.sink.RollingByTypeAndDayFileSink
agent.sinks.sink-1.sink.directory=/opt/log/data
agent.sinks.sink-1.channel = ch-1
agent.sinks.sink-1.sink.serializer.appendNewline = false

agent.sources.src-1.type = avro
agent.sources.src-1.bind = 192.168.1.130
agent.sources.src-1.port = 41414

agent.sinks.sink-1.channel = ch-1
agent.sources.src-1.channels= ch-1

程序的log4j.properties的配置：

log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname = 192.168.1.130
log4j.appender.flume.Port = 41414
log4j.appender.flume.UnsafeMode = true
log4j.appender.flume.layout=org.apache.log4j.PatternLayout
log4j.appender.flume.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %p [%c:%L] - %m%n

# configure a class's logger to output to the flume appender
log4j.logger.com.cloudera.flume.flumelog4jtest.LogTestApp = INFO,flume

项目maven的pom.xml 添加配置：

		<dependency>
			<artifactId>log4j</artifactId>
			<groupId>log4j</groupId>
			<version>1.2.16</version>
		</dependency>

		<dependency>
			<groupId>org.apache.flume.flume-ng-clients</groupId>
			<artifactId>flume-ng-log4jappender</artifactId>
			<version>1.6.0</version>
		</dependency>


		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-jdk14</artifactId>
			<version>1.7.2</version>
		</dependency>

参考资料：http://neoremind.com/2016/05/Flume+Kafka%E6%94%B6%E9%9B%86Docker%E5%AE%B9%E5%99%A8%E5%86%85%E5%88%86%E5%B8%83%E5%BC%8F%E6%97%A5%E5%BF%97%E5%BA%94%E7%94%A8%E5%AE%9E%E8%B7%B5/

转载于:https://my.oschina.net/redhat1520/blog/688326