flume-TailSource分析

最新推荐文章于 2022-12-02 23:05:42 发布

也许

最新推荐文章于 2022-12-02 23:05:42 发布

阅读量2.2k

点赞数

分类专栏： flume

本文链接：https://blog.csdn.net/cai110991/article/details/8684012

版权

本文详细分析了Flume的TailSource，解释了其作为Unix tail工具的类似功能，如何处理文件数据，以及在文件轮转时的行为。TailSource通过Cursor管理文件，监控文件变化，并通过readAllFromChannel和extractLines方法按行分割数据。在文件轮转时，TailSource会进行一系列判断以确保正确跟踪新的文件内容。

摘要由CSDN通过智能技术生成

TailSource

首先看一下tail这种source的使用接口，下面是flume-0.9.4的文档，

tail("filename"[, startFromEnd=false]{,delim="regex", delimMode="exclude|prev|next"}) :: Similar to Unix’s tail utility. One line is one event. Generates events for the entire file then stays open for more data, and follows filename. (e.g. if tailing file "foo" and then "foo" is moved to "bar" and a new file appears named "foo", it will finish reading the new "bar" file and then start from the beginning of "foo"). If the startFromEnd parameter is false, tail will re-read from the beginning of the file. If it is true, it will only start reading from the current end of file. If the last line of a file does not end with a newline character (\n), the tail source will only send an event with this last line when the tail is closed. See the section on tailing a file for details on delim and delimMode.

我们可以看出它最多可以由四个参数，最少要设置一个。

下面是TailSource的源码分析，首先看一下TailSource这个类的结构，它继承自EventSource.Base