flink(1)下载和启动

## 下载和启动

1. 安装flink需要java 8.x的环境，因此我们首先需要检查java 8是否已安装，可以通过java -version检查。

2. 下载安装包

   可以在https://flink.apache.org/downloads.html 下载对应的版本，此次我们下载 Apache Flink 1.7.2 for Scala 2.11 (asc, sha512)。

   下载后解压即可。

   $ cd ~/Downloads        # Go to download directory
   $ tar xzf flink-*.tgz   # Unpack the downloaded archive
   $ cd flink-1.7.2
   

3. 启动flink

   $ ./bin/start-cluster.sh  # Start Flink
   

   默认是单机版，启动后我们可以访问http://localhost:8081，如果页面访问正常，我们可以看到有一个可用的TaskManager实例。

   我们也可以通过$ tail log/flink-*-standalonesession-*.log 检查启动日志，确认是否启动成功。

4. 运行

   我们使用如下java代码来测试flink功能，这个任务会从socket中读取文本信息，每隔5秒统计一次每个词出现的次数。

   public class SocketWindowWordCount {
   
       public static void main(String[] args) throws Exception {
   
           // the port to connect to
           final int port;
           try {
               final ParameterTool params = ParameterTool.fromArgs(args);
               port = params.getInt("port");
           } catch (Exception e) {
               System.err.println("No port specified. Please run 'SocketWindowWordCount --port <port>'");
               return;
           }
   
           // get the execution environment
           final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
   
           // get input data by connecting to the socket
           DataStream<String> text = env.socketTextStream("localhost", port, "\n");
   
           // parse the data, group it, window it, and aggregate the counts
           DataStream<WordWithCount> windowCounts = text
               .flatMap(new FlatMapFunction<String, WordWithCount>() {
                   @Override
                   public void flatMap(String value, Collector<WordWithCount> out) {
                       for (String word : value.split("\\s")) {
                           out.collect(new WordWithCount(word, 1L));
                       }
                   }
               })
               .keyBy("word")
               .timeWindow(Time.seconds(5), Time.seconds(1))
               .reduce(new ReduceFunction<WordWithCount>() {
                   @Override
                   public WordWithCount reduce(WordWithCount a, WordWithCount b) {
                       return new WordWithCount(a.word, a.count + b.count);
                   }
               });
   
           // print the results with a single thread, rather than in parallel
           windowCounts.print().setParallelism(1);
   
           env.execute("Socket Window WordCount");
       }
   
       // Data type for words with count
       public static class WordWithCount {
   
           public String word;
           public long count;
   
           public WordWithCount() {}
   
           public WordWithCount(String word, long count) {
               this.word = word;
               this.count = count;
           }
   
           @Override
           public String toString() {
               return word + " : " + count;
           }
       }
   }
   

   由于需要监听端口，我们需要启动一个tcp服务。`

   $ nc -l 9000
   

   第二步，提交flink程序。

   $ ./bin/flink run examples/streaming/SocketWindowWordCount.jar --port 9000

   提交的程序会连接到9000端口并监听输入。

   $ nc -l 9000
   lorem ipsum
   ipsum ipsum ipsum
   bye
   

   代码中我们使用print直接输出，因此我们可以通过日志文件查看结果：

   $ tail -f log/flink-*-taskexecutor-*.out
   lorem : 1
   bye : 1
   ipsum : 4
   

   5. 停止flink

      $ ./bin/stop-cluster.sh