Flink:word count demo

flink 安装

单机安装flink非常简单，在官网下载flink,并执行安装包中bin文件下的start-cluster.sh即可，运行成功之后访问http://localhost:8081,会出现flink 的管理页面

Wordcout 代码

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;

public class WordCount {

    public static void main(String[] args) throws Exception {

        ExecutionEnvironment see = ExecutionEnvironment.getExecutionEnvironment();

        DataSet<String> text = see.fromElements("That last call is " +
            "necessary to start the actual Flink job. All operations, " +
            "such as creating sources, transformations and sinks " +
            "only build up a graph of internal operations. " +
            "Only when execute() is called is this graph of operations " +
            "thrown on a cluster or executed on your local machine.");

        DataSet<Tuple2<String,Long>> result = text.flatMap(new LineSplitter())
            .groupBy(0).sum(1);

        result.writeAsText("/Users/zhumingyuan/Documents/opensource/flink-1.6.1/output/wordcount.txt");
        //result.print();

        see.execute();
    }

    public static class LineSplitter implements FlatMapFunction<String,Tuple2<String,Long>> {
        @Override
        public void flatMap(String line, Collector<Tuple2<String, Long>> collector) throws Exception {
            String[] words = line.split(" ");
            for (String word : words) {
                collector.collect(new Tuple2(word,1L));
            }
        }
    }
}

运行样例

可以选择在本地运行代码，也可以通过管理页面提交jar进行运行。