flink写入redis（续）

参照源码：https://github.com/apache/bahir-flink/tree/master/flink-connector-redis

先分析，后修改。

flink-connector-redis代码目录：

着重看RedisSink类：

1. 包含如下几个私有对象： 

private String additionalKey;
private Integer additionalTTL;
private RedisMapper<IN> redisSinkMapper;
private RedisCommand redisCommand;
private FlinkJedisConfigBase flinkJedisConfigBase;
private RedisCommandsContainer redisCommandsContainer;
additionalKey：hset中的hashField 或 zadd中的score
additionalTTL：ttl
redisSinkMapper：redis输出映射接口，提供抽象方法，由用户实现
public interface RedisMapper<T> extends Function, Serializable {
 
    /**
     * Returns descriptor which defines data type.
     *
     * @return data type descriptor
     */
    RedisCommandDescription getCommandDescription();
 
    /**
     * Extracts key from data.
     *
     * @param data source data
     * @return key
     */
    String getKeyFromData(T data);
 
    /**
     * Extracts value from data.
     *
     * @param data source data
     * @return value
     */
    String getValueFromData(T data);
 
    /**
     * Extracts the additional key from data as an {@link Optional<String>}.
     * The default implementation returns an empty Optional.
     *
     * @param data
     * @return Optional
     */
    default Optional<String> getAdditionalKey(T data) {
        return Optional.empty();
    }
 
    /**
     * Extracts the additional time to live (TTL) for data as an {@link Optional<Integer>}.
     * The default implementation returns an empty Optional.
     *
     * @param data
     * @return Optional
     */
    default Optional<Integer> getAdditionalTTL(T data) {
        return Optional.empty();
    }
}
redisCommand：redis命令枚举类
public enum RedisCommand {
 
    /**
     * Insert the specified value at the head of the list stored at key.
     * If key does not exist, it is created as empty list before performing the push operations.
     */
    LPUSH(RedisDataType.LIST),
 
    /**
     * Insert the specified value at the tail of the list stored at key.
     * If key does not exist, it is created as empty list before performing the push operation.
     */
    RPUSH(RedisDataType.LIST),
 
    /**
     * Add the specified member to the set stored at key.
     * Specified member that is already a member of this set is ignored.
     */
    SADD(RedisDataType.SET),
 
    ...
}
flinkJedisConfigBase：redis配置信息抽象类，提供三种实现类
FlinkJedisPoolConfig：Single 
FlinkJedisClusterConfig：Cluster
FlinkJedisSentinelConfig：Sentinel
重点！​​​
RedisCommandsContainer：redis命令容器接口，提供open()、close()及hset()等抽象方法

/**
 * The container for all available Redis commands.
 */
public interface RedisCommandsContainer extends Serializable {
 
    /**
     * Open the Jedis container.
     *
     * @throws Exception if the instance can not be opened properly
     */
    void open() throws Exception;
 
    void hset(String key, String hashField, String value, Integer ttl);
    void rpush(String listName, String value);
    void lpush(String listName, String value);
    void sadd(String setName, String value);
    void publish(String channelName, String message);
    void set(String key, String value);
    void setex(String key, String value, Integer ttl);
    void pfadd(String key, String element);
    void zadd(String key, String score, String element);
    void zrem(String key, String element);
 
    /**
     * Close the Jedis container.
     *
     * @throws IOException if the instance can not be closed properly
     */
    void close() throws IOException;
}
提供两个实现类：

public class RedisContainer implements RedisCommandsContainer, Closeable {}
public class RedisClusterContainer implements RedisCommandsContainer, Closeable {}
和一个构造类：

public class RedisCommandsContainerBuilder {}
重中之重！
RedisContainer 中包含 private transient （私有、临时）的JedisPool、JedisSentinel Pool 

public class RedisContainer implements RedisCommandsContainer, Closeable {
 
    private transient JedisPool jedisPool;
    private transient JedisSentinelPool jedisSentinelPool;
 
    ...
}
RedisClusterContainer  中包含 private transient （私有、临时）的JedisCluster

public class RedisClusterContainer implements RedisCommandsContainer, Closeable {
 
    private transient JedisCluster jedisCluster;
 
    ...
}
关于 transient： 

一旦变量被transient修饰，变量将不再是对象持久化的一部分，该变量内容在序列化后无法获得访问。
transient关键字只能修饰变量，而不能修饰方法和类。注意，本地变量是不能被transient关键字修饰的。变量如果是用户自定义类变量，则该类需要实现Serializable接口。
被transient关键字修饰的变量不再能被序列化，一个静态变量不管是否被transient修饰，均不能被序列化。
解决了Jedis序列化报错的问题：

org.apache.flink.api.common.InvalidProgramException: The implementation of the RichFilterFunction is not serializable. The object probably contains or references non serializable fields.
 
...
 
Caused by: java.io.NotSerializableException: redis.clients.jedis.JedisSentinelPool
 
2.构造方法：

    public RedisSink(FlinkJedisConfigBase flinkJedisConfigBase, RedisMapper<IN> redisSinkMapper) {
        Objects.requireNonNull(flinkJedisConfigBase, "Redis connection pool config should not be null");
        Objects.requireNonNull(redisSinkMapper, "Redis Mapper can not be null");
        Objects.requireNonNull(redisSinkMapper.getCommandDescription(), "Redis Mapper data type description can not be null");
 
        this.flinkJedisConfigBase = flinkJedisConfigBase;
 
        this.redisSinkMapper = redisSinkMapper;
        RedisCommandDescription redisCommandDescription = redisSinkMapper.getCommandDescription();
 
        this.redisCommand = redisCommandDescription.getCommand();
        this.additionalTTL = redisCommandDescription.getAdditionalTTL();
        this.additionalKey = redisCommandDescription.getAdditionalKey();
    }
3.invoker()方法：

    public void invoke(IN input, Context context) throws Exception {
        String key = redisSinkMapper.getKeyFromData(input);
        String value = redisSinkMapper.getValueFromData(input);
 
        Optional<String> optAdditionalKey = redisSinkMapper.getAdditionalKey(input);
        Optional<Integer> optAdditionalTTL = redisSinkMapper.getAdditionalTTL(input);
 
        switch (redisCommand) {
            case RPUSH:
                this.redisCommandsContainer.rpush(key, value);
                break;
            case LPUSH:
                this.redisCommandsContainer.lpush(key, value);
                break;
 
            ...
 
            default:
                throw new IllegalArgumentException("Cannot process such data type: " + redisCommand);
        }
    }
4.open()和close()方法：

    @Override
    public void open(Configuration parameters) throws Exception {
        try {
            this.redisCommandsContainer = RedisCommandsContainerBuilder.build(this.flinkJedisConfigBase);
            this.redisCommandsContainer.open();
        } catch (Exception e) {
            LOG.error("Redis has not been properly initialized: ", e);
            throw e;
        }
    }
 
    @Override
    public void close() throws IOException {
        if (redisCommandsContainer != null) {
            redisCommandsContainer.close();
        }
    }
以上，

修改RedisMapper提供对应的Redis映射
修改RedisCommandsContainer提供需要的Redis命令
修改FlinkJedisConfigBase提供需要的Redis配置
根据自身需求，写一版自己的RedisSink，问题不大。