flink 本地_Flink 本地安装教程

1. 下载并安装

运行Flink之前，先确保您的java环境是Java 8 or 11，请使用java -version查看版本

$ cd ~/Downloads # Go to download directory

$ tar xzf flink-*.tgz # Unpack the downloaded archive

$ cd flink-1.10.0

启动：

$ ./bin/start-cluster.sh # Start Flink

查看日志可以看到如下信息：

$ tail log/flink-*-standalonesession-*.log

INFO ... - Rest endpoint listening at localhost:8081

INFO ... - http://localhost:8081 was granted leadership ...

INFO ... - Web frontend listening at http://localhost:8081.

INFO ... - Starting RPC endpoint for StandaloneResourceManager at akka://flink/user/resourcemanager .

INFO ... - Starting RPC endpoint for StandaloneDispatcher at akka://flink/user/dispatcher .

INFO ... - ResourceManager akka.tcp://flink@localhost:6123/user/resourcemanager was granted leadership ...

INFO ... - Starting the SlotManager.

INFO ... - Dispatcher akka.tcp://flink@localhost:6123/user/dispatcher was granted leadership ...

INFO ... - Recovering all persisted jobs.

INFO ... - Registering TaskManager ... at ResourceManager

2. 运行示例

示例代码：

public class SocketWindowWordCount {

public static void main(String[] args) throws Exception {

// the port to connect to

final int port;

try {

final ParameterTool params = ParameterTool.fromArgs(args);

port = params.getInt("port");

} catch (Exception e) {

System.err.println("No port specified. Please run 'SocketWindowWordCount --port '");

return;

}

// get the execution environment

final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

// get input data by connecting to the socket

DataStream text = env.socketTextStream("localhost", port, "\n");

// parse the data, group it, window it, and aggregate the counts

DataStream windowCounts = text

.flatMap(new FlatMapFunction() {

@Override

public void flatMap(String value, Collector out) {

for (String word : value.split("\\s")) {

out.collect(new WordWithCount(word, 1L));

}

}

})

.keyBy("word")

.timeWindow(Time.seconds(5), Time.seconds(1))

.reduce(new ReduceFunction() {

@Override

public WordWithCount reduce(WordWithCount a, WordWithCount b) {

return new WordWithCount(a.word, a.count + b.count);

}

});

// print the results with a single thread, rather than in parallel

windowCounts.print().setParallelism(1);

env.execute("Socket Window WordCount");

}

// Data type for words with count

public static class WordWithCount {

public String word;

public long count;

public WordWithCount() {}

public WordWithCount(String word, long count) {

this.word = word;

this.count = count;

}

@Override

public String toString() {

return word + " : " + count;

}

}

}

首先先使用netcat打开本地服务

$ nc -l 9000

提交示例程序

$ ./bin/flink run examples/streaming/SocketWindowWordCount.jar --port 9000

Starting execution of program

程序连接到socket，并等待输入，可以在web端查看程序是否符合预期效果

开始输入数据，测试程序效果：

$ nc -l 9000

lorem ipsum

ipsum ipsum ipsum

bye

输出：

$ tail -f log/flink-*-taskexecutor-*.out

lorem : 1

bye : 1

ipsum : 4