Interceptor概述
- 拦截器可以拦截Event,允许或不允许Event通过,或在允许通过的时,改变Event内容,这种改变包括改变Event的体或头信息。
- 拦截器可以手动开发,只要实现org.apache.flume.interceptor.Interceptor接口,在其中编写拦截规则即可。
- Flume也内置了很多拦截器,可以直接使用。
- 可以同时配置多 个拦截器,依次拦截Event处理,即,采用了责任链模式。
Timestamp Interceptor
This interceptor inserts into the event headers, the time in millis at which it processes the event. This interceptor inserts a header with key timestamp
(or as specified by the header
property) whose value is the relevant timestamp. This interceptor can preserve an existing timestamp if it is already present in the configuration.
Property Name | Default | Description |
---|---|---|
type | – | The component type name, has to be timestamp or the FQCN |
headerName | timestamp | The name of the header in which to place the generated timestamp. |
preserveExisting | false | If the timestamp already exists, should it be preserved - true or false |
Example for agent named a1:
a1.sources.s1.interceptors.i1.type = timestamp
时间戳过滤,在event的head中添加时间戳
Host Interceptor
This interceptor inserts the hostname or IP address of the host that this agent is running on. It inserts a header with key host
or a configured key whose value is the hostname or IP address of the host, based on configuration.
Property Name | Default | Description |
---|---|---|
type | – | The component type name, has to be host |
preserveExisting | false | If the host header already exists, should it be preserved - true or false |
useIP | true | Use the IP Address if true, else use hostname. |
hostHeader | host | The header key to be used. |
Example for agent named a1:
a1.sources.s1.interceptors.i1.type = host
a1.sources.s1.interceptors.i1.useIP=false
主机名过滤,在event的head中添加主机名
Static Interceptor
Static interceptor allows user to append a static header with static value to all events.
The current implementation does not allow specifying multiple headers at one time. Instead user might chain multiple static interceptors each defining one static header.
Property Name | Default | Description |
---|---|---|
type | – | The component type name, has to be static |
preserveExisting | true | If configured header already exists, should it be preserved - true or false |
key | key | Name of header that should be created |
value | value | Static value that should be created |
Example for agent named a1:
a1.sources.r1.interceptors.i1.type = static
a1.sources.r1.interceptors.i1.key = datacenter
a1.sources.r1.interceptors.i1.value = NEW_YORK
自定义静态过滤,在event的head中添加一对自定义的key-value
Regex Filtering Interceptor
正则过滤拦截器,拦截下Event之后,利用正则匹配日志的体,根据是否匹配决定是否保留或是否取出当前Event。
This interceptor filters events selectively by interpreting the event body as text and matching the text against a configured regular expression. The supplied regular expression can be used to include events or exclude events.
此拦截器通过将事件主体解释为文本并将文本与配置的正则表达式进行匹配来有选择地过滤事件。 提供的正则表达式可用于包括事件或排除事件。
Property Name | Default | Description |
---|---|---|
type | – | The component type name has to be regex_filter |
regex | ”.*” | Regular expression for matching against events |
excludeEvents | false | If true, regex determines events to exclude, otherwise regex determines events to include. |
a1.sources.s1.interceptors.i2.type = regex_filter
a1.sources.s1.interceptors.i2.regex= ^\\{.*\\}$
正则过滤,过滤event的body中的内容,保留符合正则表达式的
Search and Replace Interceptor
搜索和替换拦截器,拦截下Event后,通过正则匹配日志中的体,将符合正则的部分替换为指定的内容。