flink将数据写入redis

前言

通过flink分析出来的数据，经常需要写入到外部存储系统，以供其他应用来使用这些数据，比如redis就是一个很好的存储数据库，写入到redis后，应用可以读取redis中的数据做统计、分析、计算等

1、导入maven依赖

        <dependency>
            <groupId>org.apache.bahir</groupId>
            <artifactId>flink-connector-redis_2.11</artifactId>
            <version>1.0</version>
        </dependency>

2、核心代码

import com.congge.source.SensorReading;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.redis.RedisSink;
import org.apache.flink.streaming.connectors.redis.common.config.FlinkJedisPoolConfig;
import org.apache.flink.streaming.connectors.redis.common.mapper.RedisCommand;
import org.apache.flink.streaming.connectors.redis.common.mapper.RedisCommandDescription;
import org.apache.flink.streaming.connectors.redis.common.mapper.RedisMapper;

public class SinkRedis {

    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setParallelism(1);

        //从环境的集合中获取数据
        String path = "E:\code-self\flink_study\src\main\resources\sensor.txt";
        DataStreamSource<String> inputStream = env.readTextFile(path);

        SingleOutputStreamOperator<SensorReading> dataStream = inputStream.map(new MapFunction<String, SensorReading>() {
            @Override
            public SensorReading map(String value) throws Exception {
                String[] fields = value.split(",");
                return new SensorReading(fields[0], new Long(fields[1]), new Double(fields[2]));
            }
        });

        FlinkJedisPoolConfig.Builder builder = new FlinkJedisPoolConfig.Builder();
        builder.setHost("localhost");
        builder.setPort(6379);
        FlinkJedisPoolConfig build = builder.build();

        dataStream.addSink(new RedisSink<>(build, new RedisMapper<SensorReading>() {
            @Override
            public RedisCommandDescription getCommandDescription() {
                return new RedisCommandDescription(RedisCommand.HSET,"sensor_temp");
            }

            @Override
            public String getKeyFromData(SensorReading data) {
                return data.getId();
            }

            @Override
            public String getValueFromData(SensorReading data) {
                return data.getTemperature().toString();
            }
        }));

        env.execute();

    }

}

最关键的就是通过dataStream.addSink 这个方法，将在这个方法中，其实就需要传入一个实现了SinkFunction的类即可，比如这里的：RedisMapper，有兴趣的同学可以点进去一探究竟

3、启动本地的redis服务，并运行上面的代码

可以发现，日志中的数据通过程序最终存储到redis库中