Flume使用笔记

1、当一台机器运行多个flume脚本，时执行:ps -aux|grep flume会输出很多个进程出来，且当本地的环境变量配置很多时（如：Hadoop，Hbase...），此时会在控制台打印很多JVM加载的依赖，没办法看哪个进程运行了哪个flume脚本。此时可以根据端口来找：

（1）netstat -nlpt|grep 44444

（2）第（1）步可以看到对应的进程号，只需要将该进程号kill掉即可。

2、当在采集端配置channel.type=file，source有多个的时候

nginx.channels.channel0.type = file
nginx.channels.channel0.capacity = 1000000
nginx.channels.channel0.batch-size=1000
nginx.channels.channel0.transactionCapacity = 100000
nginx.channels.channel0.keep-alive = 20
nginx.channels.channel0.request-timeout=50000
nginx.channels.channel0.connect-timeout=50000

会报如下错误：

23 Apr 2015 16:58:54,058 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to deliver event. Exception follows.
java.lang.IllegalStateException: Channel closed [channel=channel4]. Due to java.lang.IllegalArgumentException: CheckpointDir /root/.flume/file-channel/checkpoint could not be created
 at org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:352)
 at org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)

解决办法：把file改成memoery的方式了(但这种方式会丢数据)

nginx.channels.memoryChannel.type = memory
nginx.channels.memoryChannel.capacity = 10000
nginx.channels.memoryChannel.transactionCapacity = 10000