目的:把windows里iis日志,通过flume实时传输到linux中的hadoop集群分布式存储
windows中日志生成在C:\inetpub\logs\LogFiles\W3SVC1,传输到Linux中的HDFS
流程:
1.windows、linux中安装好flume
2.配置信息
windows中配置信息:
a1.sources = r1
a1.sinks = k1
a1.channels = c1
a1.sources.r1.type = spooldir #spooldir Source用来监控目标文件夹内文件的更改
a1.sources.r1.channels = c1
a1.sources.r1.spoolDir = C:\\inetpub\\logs\\LogFiles\\W3SVC1
a1.sources.r1.fileHeader = true
a1.sources.r1.deserializer.maxLineLength = 5120
a1.sources.r1.basenameHeader = true
a1.sources.r1.basenameHeaderKey = fileName
a1.sources.r1.ignorePattern = ^(.