1 flink开发环境安装
下载git clone https://github.com/apache/flink
1.1 cmd命令行执行:
set MAVEN_OPTS="-Xmx4G"
mvn clean install package -Dmaven.test.skip=true
1.2 powershell命令行执行
set MAVEN_OPTS="-Xmx4G"
mvn clean install package ‘-Dmaven.test.skip=true’
1.3 flink的目录结构
cd E:\app-installtools\flink\flink-dist\target\flink-1.9-SNAPSHOT-bin\flink-1.9-SNAPSHOT\bin
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 2019/4/24 11:50 29647 config.sh
-a---- 2019/4/24 11:50 2279 flink
-a---- 2019/4/24 11:50 2847 flink-console.sh
-a---- 2019/4/24 11:50 6545 flink-daemon.sh
-a---- 2019/4/24 11:50 1271 flink.bat
-a---- 2019/4/24 11:50 1603 historyserver.sh
-a---- 2019/4/24 11:50 2967 jobmanager.sh
-a---- 2019/4/24 11:50 1849 mesos-appmaster-job.sh
-a---- 2019/4/24 11:50 1883 mesos-appmaster.sh
-a---- 2019/4/24 11:50 1935 mesos-taskmanager.sh
-a---- 2019/4/24 11:50 1207 pyflink-stream.sh
-a---- 2019/4/24 11:50 1166 pyflink.bat
-a---- 2019/4/24 11:50 1132 pyflink.sh
-a---- 2019/4/24 11:50 3517 sql-client.sh
-a---- 2019/4/24 11:50 2597 standalone-job.sh
-a---- 2019/4/24 11:50 3364 start-cluster.bat
-a---- 2019/4/24 11:50 1889 start-cluster.sh
-a---- 2019/4/24 11:50 3538 start-scala-shell.sh
-a---- 2019/4/24 11:50 1900 start-zookeeper-quorum.sh
-a---- 2019/4/24 11:50 1663 stop-cluster.sh
-a---- 2019/4/24 11:50 1891 stop-zookeeper-quorum.sh
-a---- 2019/4/24 11:50 3941 taskmanager.sh
-a---- 2019/4/24 11:50 1714 yarn-session.sh
-a---- 2019/4/24 11:50 2346 zookeeper.sh
1.4 启动flink
E:\app-installtools\flink\flink-dist\target\flink-1.9-SNAPSHOT-bin\flink-1.9-SNAPSHOT\bin\start-cluster.bat
Web interface by default on http://localhost:8081/.
2 flink计算word count案例
2.1 word count案例1
from flink.plan.Environment import get_environment
from flink.functions.GroupReduceFunction import GroupReduceFunction
class Adder(GroupReduceFunction):
def reduce(self, iterator, collector):
count, word &#