Flume(三十一)Custom Interceptor

在Flume中，也允许自定义拦截器。但是不同于其他组件，自定义Interceptor的时候，需要再额外覆盖其中的内部接口。

构建Maven工程，导入对应的依赖。

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.example</groupId>
    <artifactId>flume</artifactId>
    <version>1.0-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-core</artifactId>
            <version>1.11.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-sdk</artifactId>
            <version>1.11.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-configuration</artifactId>
            <version>1.11.0</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                    <encoding>UTF-8</encoding>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

自定义类实现Interceptor接口，覆盖其中initialize，intercept和close方法

定义静态内部类，实现Interceptor.Builder内部接口

package com.st;

import org.apache.flume.Context;
import org.apache.flume.Event;
import org.apache.flume.interceptor.Interceptor;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
/**
 * 模拟timestamp  Interceptor
 */
public class SelfInterceptor  implements Interceptor {
    @Override
    public void initialize() {
    }
    //对event对象的处理就放在这个方法中
    @Override
    public Event intercept(Event event) {
        //时间戳在head中，首先获取时间戳
        Map<String, String> headers = event.getHeaders();
        //判断是否原来制定
        if(headers.containsKey("time")||headers.containsKey("timestamp"))
        {
            //如果制定，直接返回
            return event;
        }else{
            headers.put("timestamp",System.currentTimeMillis()+"");
            event.setHeaders(headers);
        }
        //下面这个例子是过滤掉hello这个字符串
            // 获取事件数据
       /* String eventData = new String(event.getBody(), StandardCharsets.UTF_8);
        // 检查事件数据中是否包含指定字符串
        if (eventData.contains("hello")) {
            // 如果包含指定字符串，则过滤掉该事件，返回 null
            return null;
        }*/

        return event;
    }

    @Override
    public List<Event> intercept(List<Event> events) {
        //处理拦截之后的events
        List<Event> interceptedEvents = new ArrayList<>();

        for (Event event : events) {
            Event interceptedEvent = intercept(event);
            if (interceptedEvent != null) {
                interceptedEvents.add(interceptedEvent);
            }
        }
        return interceptedEvents;
    }

    @Override
    public void close() {
    }
    //覆盖内部接口
    public static class Builder implements Interceptor.Builder{
        //产生要使用的拦截器对象
        @Override
        public Interceptor build() {
            return new SelfInterceptor();
        }
        @Override
        public void configure(Context context) {
        }
    }
}

打成jar包方法Flume安装目录的lib目录下

编写配置文件selfinterceptor.conf

a1.sources = s1

a1.channels = c1

a1.sinks = k1

a1.sources.s1.type = netcat

a1.sources.s1.bind = m1

a1.sources.s1.port = 9999

# 指定拦截器

a1.sources.s1.interceptors = i1

a1.sources.s1.interceptors.i1.type = com.st.SelfInterceptor$Builder

a1.channels.c1.type = memory

a1.sinks.k1.type = logger

a1.sources.s1.channels = c1

a1.sinks.k1.channel = c1

启动flume

flume-ng agent -c $FLUME_HOME/conf -n a1  -f selfinterceptor.conf  -Dflume.root.logger=INFO,console

发送数据nc m1 9999

查看数据，发现有了timestamp