HDFS Connector

示例:

public static void main(String[] args) throws Exception {

        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        String sourceFilePath = "D:\\\\yly\\\\BaiduNetdiskDownload\\\\最全最新flink教程\\\\000.代码+环境\\\\00.flink-train-master\\\\flink-train\\\\data\\\\04\\\\hello1.txt";

        DataStream<String> data = env.socketTextStream("192.168.1.176", 9999);

        data.print().setParallelism(1);

//        String filePath = "hdfs://10.0.0.47:8020/user/testdata/";

        String filePath = "D:\\yly\\BaiduNetdiskDownload\\最全最新flink教程\\000.代码+环境\\00.flink-train-master\\flink-train\\data\\04"; //此处内容为目录不是文件

        BucketingSink<String> sink = new BucketingSink<String>(filePath);

        sink.setBucketer(new DateTimeBucketer<String>("yyyy-MM-dd--HHmm"));//sink出去的文件的名字

        sink.setWriter(new StringWriter<String>());

//        sink.setBatchSize(1024 * 1024 * 400); // this is 400 MB,

//        sink.setBatchRolloverInterval(20 * 60 * 1000); // this is 20 mins

        sink.setBatchRolloverInterval(2000);

        data.addSink(sink);

        env.execute("FileSystemSinkApp");

    }

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值