Apache Flink 简单易懂渐进式学习教程(一)： 单节点安装教程和验证

 

目录

一，准备工作

二，安装

三，启动单节点flink

四，验证

五，关闭单节点flink

---

 

 

一，准备工作

1，jdk的安装和验证

Flink可在Linux，Mac OS X和Windows上运行。为了能够运行Flink，要求安装Java 8.x。

安装教程：https://www.t9vg.com/archives/346

通过以下命令来检查Java的正确安装：

java -version

输出如下则安装正确：

java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

2，下载flink安装包

官网地址：https://flink.apache.org/downloads.html

本文使用：Apache Flink 1.8.1 for Scala 2.11

二，安装

$ cd ~/Downloads        # 进入下载文件的目录
$ tar xzf flink-*.tgz   # 解压文件
$ cd flink-1.8.0

三，启动单节点flink

$ ./bin/start-cluster.sh  # 启动 Flink

访问localhost:8081，确认监控界面正常启动，Web监控界面会显示有一个可用的TaskManager实例。

 

您还可以通过检查logs目录中的日志文件来验证系统是否正在运行：

$ tail log/flink-*-standalonesession-*.log
INFO ... - Rest endpoint listening at localhost:8081
INFO ... - http://localhost:8081 was granted leadership ...
INFO ... - Web frontend listening at http://localhost:8081.
INFO ... - Starting RPC endpoint for StandaloneResourceManager at akka://flink/user/resourcemanager .
INFO ... - Starting RPC endpoint for StandaloneDispatcher at akka://flink/user/dispatcher .
INFO ... - ResourceManager akka.tcp://flink@localhost:6123/user/resourcemanager was granted leadership ...
INFO ... - Starting the SlotManager.
INFO ... - Dispatcher akka.tcp://flink@localhost:6123/user/dispatcher was granted leadership ...
INFO ... - Recovering all persisted jobs.
INFO ... - Registering TaskManager ... under ... at the SlotManager.

 

 

四，验证

本文将验证官方给出的SocketWindowWordCount代码，代码将从socket监听中读取数据，并且每5秒打印一次前5秒内每个不同单词的出现次数，即使用windos的滚动时间窗口。

SocketWindowWordCount的scala源码如下：

object SocketWindowWordCount {

    def main(args: Array[String]) : Unit = {

        // the port to connect to
        val port: Int = try {
            ParameterTool.fromArgs(args).getInt("port")
        } catch {
            case e: Exception => {
                System.err.println("No port specified. Please run 'SocketWindowWordCount --port <port>'")
                return
            }
        }

        // get the execution environment
        val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment

        // get input data by connecting to the socket
        val text = env.socketTextStream("localhost", port, '\n')

        // parse the data, group it, window it, and aggregate the counts
        val windowCounts = text
            .flatMap { w => w.split("\\s") }
            .map { w => WordWithCount(w, 1) }
            .keyBy("word")
            .timeWindow(Time.seconds(5), Time.seconds(1))
            .sum("count")

        // print the results with a single thread, rather than in parallel
        windowCounts.print().setParallelism(1)

        env.execute("Socket Window WordCount")
    }

    // Data type for words with count
    case class WordWithCount(word: String, count: Long)
}

1，首先，我们使用netcat来启动本地服务器

$ nc -l 9000

2，提交Flink计划：

$ ./bin/flink run examples/streaming/SocketWindowWordCount.jar --port 9000
Starting execution of program

此时程序等待socket输入。可以在web监控页面查看作业启动情况：

3，在nc窗口输入数据：

$ nc -l 9000
lorem ipsum
ipsum ipsum ipsum
bye

4，查看计算结果

$ tail -f log/flink-*-taskexecutor-*.out
lorem : 1
bye : 1
ipsum : 4

五，关闭单节点flink

$ ./bin/stop-cluster.sh   # 关闭 Flink

参考：https://ci.apache.org/projects/flink/flink-docs-stable/ 

参考：https://www.t9vg.com/archives/457