Flume 配置和读取tomcat文件到指定位置

需要交流请进群-- 494831900 --我和群中朋友会及时回答

1：去官网下载 flume 1.5  网址是：flume.apache.org

2: 解压对应的tar 包 

3：进去flume的conf目录 创建一个文件 vi  example.conf 

# Define a memory channel called ch1 on agent1

agent1.channels.ch1.type = memory

 

# Define an Avro source called avro-source1 on agent1 and tell it

# to bind to  0.0 . 0.0 : 41414 . Connect it to channel ch1.

agent1.sources.avro-source1.channels = ch1

agent1.sources.avro-source1.type = avro

agent1.sources.avro-source1.bind =  0.0 . 0.0

agent1.sources.avro-source1.port =  41414

 

# Define a logger sink that simply logs all events it receives

# and connect it to the other end of the same channel.

agent1.sinks.log-sink1.channel = ch1

agent1.sinks.log-sink1.type = logger

 

# Finally, now that we've defined all of our components, tell

# agent1 which ones we want to activate.

agent1.channels = ch1

agent1.sources = avro-source1

agent1.sinks = log-sink1

_____________________________________________________________

把内容拷贝进去

 mv  flume-env.sh.template   flume-env.sh

然后更改  flume-env.sh 里面的java_home 变成你自己的 

执行命令  --进入bin目录后 如果配置了环境变量 不需要进去bin目录执行次命令否则进入bin执行此命令

flume-ng agent --conf conf/ -f example.conf  -Dflume.root.logger=DEBUG,console -n agent1

--name,-n <agentname>
是指配置文件中的 agent的名字 我这个里叫 agent1

--conf-file,-f <file>  这个是指配置文件的目录 和要指定的配置文件

flume-ng avro-client --conf /usr/local/flume/conf -H localhost -p  41414   -F /etc/passwd -Dflume.root.logger=DEBUG,console

会打印出内容

----------------------读取tomcat logs ------------------------------------------------

在配置文件中 新建一个 tomcat.conf 内容如下

agent1.channels.ch1.type = file
 
agent1.sources.avro-source1.channels = ch1
agent1.sources.avro-source1.type = exec
agent1.sources.avro-source1.bind = 0.0.0.0
agent1.sources.avro-source1.port = 41414
agent1.sources.avro-source1.command = t ail -n +0 -F /usr/local/tomcat/logs/catalina.out  #linux命令的输出接受  
 
agent1.sinks.log-sink1.channel = ch1
agent1.sinks.log-sink1.type = file_roll   #是指输入类型 读取文件时的
agent1.sinks.log-sink1.directory=/var/log/d ata     #这个是指读取tomcat的日志到什么目录下
 
agent1.channels = ch1
agent1.sources = avro-source1
agent1.sinks = log-sink1a         

配置好后，执行命令：flume-ng agent --conf conf --conf-file conf/tomcat.conf --name agent1 -Dflume.root.logger=INFO,console

启动tomcat  进入 cd  /var/log/d at a 就会看到如下图

大功告成！！！

下面是一篇详细介绍flume的文章链接

http://www.jb51.net/article/53542.htm