Flume Interceptor

Interceptor概述

• 拦截器可以拦截Event，允许或不允许Event通过，或在允许通过的时，改变Event内容，这种改变包括改变Event的体或头信息。
• 拦截器可以手动开发，只要实现org.apache.flume.interceptor.Interceptor接口，在其中编写拦截规则即可。
• Flume也内置了很多拦截器，可以直接使用。
• 可以同时配置多 个拦截器，依次拦截Event处理，即，采用了责任链模式。

Timestamp Interceptor

This interceptor inserts into the event headers, the time in millis at which it processes the event. This interceptor inserts a header with key timestamp (or as specified by the header property) whose value is the relevant timestamp. This interceptor can preserve an existing timestamp if it is already present in the configuration.

Property Name	Default	Description
type	–	The component type name, has to be timestamp or the FQCN
headerName	timestamp	The name of the header in which to place the generated timestamp.
preserveExisting	false	If the timestamp already exists, should it be preserved - true or false

Example for agent named a1:

  a1.sources.s1.interceptors.i1.type = timestamp

时间戳过滤，在event的head中添加时间戳

Host Interceptor

This interceptor inserts the hostname or IP address of the host that this agent is running on. It inserts a header with key host or a configured key whose value is the hostname or IP address of the host, based on configuration.

Property Name	Default	Description
type	–	The component type name, has to be host
preserveExisting	false	If the host header already exists, should it be preserved - true or false
useIP	true	Use the IP Address if true, else use hostname.
hostHeader	host	The header key to be used.

Example for agent named a1:

a1.sources.s1.interceptors.i1.type = host
a1.sources.s1.interceptors.i1.useIP=false

主机名过滤，在event的head中添加主机名

Static Interceptor

Static interceptor allows user to append a static header with static value to all events.

The current implementation does not allow specifying multiple headers at one time. Instead user might chain multiple static interceptors each defining one static header.

Property Name	Default	Description
type	–	The component type name, has to be static
preserveExisting	true	If configured header already exists, should it be preserved - true or false
key	key	Name of header that should be created
value	value	Static value that should be created

Example for agent named a1:

a1.sources.r1.interceptors.i1.type = static
a1.sources.r1.interceptors.i1.key = datacenter
a1.sources.r1.interceptors.i1.value = NEW_YORK

自定义静态过滤，在event的head中添加一对自定义的key-value

Regex Filtering Interceptor

正则过滤拦截器，拦截下Event之后，利用正则匹配日志的体，根据是否匹配决定是否保留或是否取出当前Event。
This interceptor filters events selectively by interpreting the event body as text and matching the text against a configured regular expression. The supplied regular expression can be used to include events or exclude events.
此拦截器通过将事件主体解释为文本并将文本与配置的正则表达式进行匹配来有选择地过滤事件。 提供的正则表达式可用于包括事件或排除事件。

Property Name	Default	Description
type	–	The component type name has to be regex_filter
regex	”.*”	Regular expression for matching against events
excludeEvents	false	If true, regex determines events to exclude, otherwise regex determines events to include.

a1.sources.s1.interceptors.i2.type = regex_filter
a1.sources.s1.interceptors.i2.regex= ^\\{.*\\}$

正则过滤，过滤event的body中的内容，保留符合正则表达式的

Search and Replace Interceptor

搜索和替换拦截器，拦截下Event后，通过正则匹配日志中的体，将符合正则的部分替换为指定的内容。