Flink源码编译(win10)

在这里插入图片描述

Flink源码编译

step1:环境准备

win10

maven3.2.5(官方推荐)

jdk8

node.js16.6.2

git2.32.0

Flink-src

step2:Flink源码获取

1.github拉取

git clone https://github.com/apache/flink.git

2.官网下载

step3:maven安装

重点:配置阿里云国内镜像

<mirror>
  <id>aliyunmaven</id>
  <mirrorOf>*</mirrorOf>
  <name>阿里云公共仓库</name>
  <url>https://maven.aliyun.com/repository/public</url>
</mirror>

step4:编译flink-runtime-web中web-dashboard

由于flink-runtime-web模块在编译时经常卡住报错,所以先把该模块手动编译了,后边整体编译就顺利了。

  1. 安装node.js

  2. 注释插件com.github.eirslett

    <!--            <plugin>-->
    <!--                <groupId>com.github.eirslett</groupId>-->
    <!--                <artifactId>frontend-maven-plugin</artifactId>-->
    <!--                <version>1.6</version>-->
    <!--                <executions>-->
    <!--                    <execution>-->
    <!--                        <id>install node and npm</id>-->
    <!--                        <goals>-->
    <!--                            <goal>install-node-and-npm</goal>-->
    <!--                        </goals>-->
    <!--                        <configuration>-->
    <!--                            <nodeVersion>v10.9.0</nodeVersion>-->
    <!--                        </configuration>-->
    <!--                    </execution>-->
    <!--                    <execution>-->
    <!--                        <id>npm install</id>-->
    <!--                        <goals>-->
    <!--                            <goal>npm</goal>-->
    <!--                        </goals>-->
    <!--                        <configuration>-->
    <!--                            <arguments>ci &#45;&#45;cache-max=0 &#45;&#45;no-save</arguments>-->
    <!--                            <environmentVariables>-->
    <!--                                <HUSKY_SKIP_INSTALL>true</HUSKY_SKIP_INSTALL>-->
    <!--                            </environmentVariables>-->
    <!--                        </configuration>-->
    <!--                    </execution>-->
    <!--                    <execution>-->
    <!--                        <id>npm run build</id>-->
    <!--                        <goals>-->
    <!--                            <goal>npm</goal>-->
    <!--                        </goals>-->
    <!--                        <configuration>-->
    <!--                            <arguments>run build</arguments>-->
    <!--                        </configuration>-->
    <!--                    </execution>-->
    <!--                </executions>-->
    <!--                <configuration>-->
    <!--                    <workingDirectory>web-dashboard</workingDirectory>-->
    <!--                </configuration>-->
    <!--            </plugin>-->
    
  3. 进入flink-runtime-web/web-dashboard文件夹路径下
    npm install
    
    npm run build
    

step5: 整体编译Flink

  • 如果你需要使用指定hadoop的版本,可以通过指定“-Dhadoop.version”来设置,编译命令如下:
mvn clean install -DskipTests -Dhadoop.version=2.6.0
# 或者
mvn clean package -DskipTests -Dhadoop.version=2.6.1
  • 官方推荐
mvn clean install -DskipTests
  • 自用

    mvn clean install -T 2C -Dfast -Dmaven.compile.fork=true -DskipTests -Dscala-2.11 -X -Drat.skip=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true
    
  • 成功标志

    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Summary:
    [INFO]
    [INFO] force-shading ...................................... SUCCESS [  1.700 s]
    [INFO] flink .............................................. SUCCESS [  1.762 s]
    [INFO] flink-annotations .................................. SUCCESS [  3.642 s]
    [INFO] flink-shaded-curator ............................... SUCCESS [  3.884 s]
    [INFO] flink-test-utils-parent ............................ SUCCESS [  1.998 s]
    [INFO] flink-test-utils-junit ............................. SUCCESS [  3.384 s]
    [INFO] flink-metrics ...................................... SUCCESS [  2.399 s]
    [INFO] flink-metrics-core ................................. SUCCESS [  2.330 s]
    [INFO] flink-core ......................................... SUCCESS [ 15.151 s]
    [INFO] flink-java ......................................... SUCCESS [  6.007 s]
    [INFO] flink-queryable-state .............................. SUCCESS [  1.646 s]
    [INFO] flink-queryable-state-client-java .................. SUCCESS [  2.140 s]
    [INFO] flink-filesystems .................................. SUCCESS [  1.577 s]
    [INFO] flink-hadoop-fs .................................... SUCCESS [  3.696 s]
    [INFO] flink-runtime ...................................... SUCCESS [ 46.766 s]
    [INFO] flink-scala ........................................ SUCCESS [ 50.829 s]
    [INFO] flink-mapr-fs ...................................... SUCCESS [  2.938 s]
    [INFO] flink-filesystems :: flink-fs-hadoop-shaded ........ SUCCESS [  4.855 s]
    [INFO] flink-s3-fs-base ................................... SUCCESS [  3.841 s]
    [INFO] flink-s3-fs-hadoop ................................. SUCCESS [ 12.073 s]
    [INFO] flink-s3-fs-presto ................................. SUCCESS [ 16.781 s]
    [INFO] flink-swift-fs-hadoop .............................. SUCCESS [ 25.736 s]
    [INFO] flink-oss-fs-hadoop ................................ SUCCESS [ 23.457 s]
    [INFO] flink-azure-fs-hadoop .............................. SUCCESS [ 16.745 s]
    [INFO] flink-optimizer .................................... SUCCESS [  4.232 s]
    [INFO] flink-clients ...................................... SUCCESS [  2.655 s]
    [INFO] flink-streaming-java ............................... SUCCESS [ 12.647 s]
    [INFO] flink-test-utils ................................... SUCCESS [  7.587 s]
    [INFO] flink-runtime-web .................................. SUCCESS [ 18.953 s]
    [INFO] flink-examples ..................................... SUCCESS [  0.804 s]
    [INFO] flink-examples-batch ............................... SUCCESS [ 24.864 s]
    [INFO] flink-connectors ................................... SUCCESS [  1.650 s]
    [INFO] flink-hadoop-compatibility ......................... SUCCESS [ 12.400 s]
    [INFO] flink-state-backends ............................... SUCCESS [  1.502 s]
    [INFO] flink-statebackend-rocksdb ......................... SUCCESS [  6.749 s]
    [INFO] flink-tests ........................................ SUCCESS [ 48.688 s]
    [INFO] flink-streaming-scala .............................. SUCCESS [ 49.604 s]
    [INFO] flink-table ........................................ SUCCESS [  1.497 s]
    [INFO] flink-table-common ................................. SUCCESS [  5.309 s]
    [INFO] flink-table-api-java ............................... SUCCESS [  4.980 s]
    [INFO] flink-table-api-java-bridge ........................ SUCCESS [  5.868 s]
    [INFO] flink-table-api-scala .............................. SUCCESS [ 16.020 s]
    [INFO] flink-table-api-scala-bridge ....................... SUCCESS [ 15.915 s]
    [INFO] flink-sql-parser ................................... SUCCESS [  8.971 s]
    [INFO] flink-libraries .................................... SUCCESS [  1.644 s]
    [INFO] flink-cep .......................................... SUCCESS [ 15.363 s]
    [INFO] flink-table-planner ................................ SUCCESS [03:10 min]
    [INFO] flink-table-runtime-blink .......................... SUCCESS [  7.664 s]
    [INFO] flink-table-planner-blink .......................... SUCCESS [04:10 min]
    [INFO] flink-jdbc ......................................... SUCCESS [  3.730 s]
    [INFO] flink-hbase ........................................ SUCCESS [ 18.697 s]
    [INFO] flink-hcatalog ..................................... SUCCESS [ 11.274 s]
    [INFO] flink-metrics-jmx .................................. SUCCESS [  5.997 s]
    [INFO] flink-formats ...................................... SUCCESS [  1.689 s]
    [INFO] flink-json ......................................... SUCCESS [  8.715 s]
    [INFO] flink-connector-kafka-base ......................... SUCCESS [  5.394 s]
    [INFO] flink-connector-kafka-0.9 .......................... SUCCESS [  6.082 s]
    [INFO] flink-connector-kafka-0.10 ......................... SUCCESS [  3.663 s]
    [INFO] flink-connector-kafka-0.11 ......................... SUCCESS [  5.391 s]
    [INFO] flink-connector-elasticsearch-base ................. SUCCESS [ 13.223 s]
    [INFO] flink-connector-elasticsearch2 ..................... SUCCESS [ 20.612 s]
    [INFO] flink-connector-elasticsearch5 ..................... SUCCESS [ 21.846 s]
    [INFO] flink-connector-elasticsearch6 ..................... SUCCESS [  9.336 s]
    [INFO] flink-connector-elasticsearch7 ..................... SUCCESS [  9.337 s]
    [INFO] flink-orc .......................................... SUCCESS [ 11.241 s]
    [INFO] flink-csv .......................................... SUCCESS [  2.528 s]
    [INFO] flink-connector-hive ............................... SUCCESS [ 28.282 s]
    [INFO] flink-connector-rabbitmq ........................... SUCCESS [  5.132 s]
    [INFO] flink-connector-twitter ............................ SUCCESS [  7.336 s]
    [INFO] flink-connector-nifi ............................... SUCCESS [  2.354 s]
    [INFO] flink-connector-cassandra .......................... SUCCESS [ 16.468 s]
    [INFO] flink-avro ......................................... SUCCESS [ 13.278 s]
    [INFO] flink-connector-filesystem ......................... SUCCESS [ 10.212 s]
    [INFO] flink-connector-kafka .............................. SUCCESS [  6.486 s]
    [INFO] flink-connector-gcp-pubsub ......................... SUCCESS [  9.487 s]
    [INFO] flink-connector-kinesis ............................ SUCCESS [ 11.882 s]
    [INFO] flink-sql-connector-elasticsearch7 ................. SUCCESS [ 11.679 s]
    [INFO] flink-sql-connector-elasticsearch6 ................. SUCCESS [ 10.815 s]
    [INFO] flink-sql-connector-kafka-0.9 ...................... SUCCESS [  1.615 s]
    [INFO] flink-sql-connector-kafka-0.10 ..................... SUCCESS [  1.722 s]
    [INFO] flink-sql-connector-kafka-0.11 ..................... SUCCESS [  2.321 s]
    [INFO] flink-sql-connector-kafka .......................... SUCCESS [  2.067 s]
    [INFO] flink-connector-kafka-0.8 .......................... SUCCESS [  5.224 s]
    [INFO] flink-avro-confluent-registry ...................... SUCCESS [  4.200 s]
    [INFO] flink-parquet ...................................... SUCCESS [  9.858 s]
    [INFO] flink-sequence-file ................................ SUCCESS [  2.365 s]
    [INFO] flink-compress ..................................... SUCCESS [  6.189 s]
    [INFO] flink-examples-streaming ........................... SUCCESS [ 21.874 s]
    [INFO] flink-examples-table ............................... SUCCESS [ 13.277 s]
    [INFO] flink-examples-build-helper ........................ SUCCESS [  0.635 s]
    [INFO] flink-examples-streaming-twitter ................... SUCCESS [  1.447 s]
    [INFO] flink-examples-streaming-state-machine ............. SUCCESS [  1.183 s]
    [INFO] flink-examples-streaming-gcp-pubsub ................ SUCCESS [  5.948 s]
    [INFO] flink-container .................................... SUCCESS [  5.415 s]
    [INFO] flink-queryable-state-runtime ...................... SUCCESS [  6.685 s]
    [INFO] flink-end-to-end-tests ............................. SUCCESS [  1.505 s]
    [INFO] flink-cli-test ..................................... SUCCESS [  3.998 s]
    [INFO] flink-parent-child-classloading-test-program ....... SUCCESS [  4.254 s]
    [INFO] flink-parent-child-classloading-test-lib-package ... SUCCESS [  3.488 s]
    [INFO] flink-dataset-allround-test ........................ SUCCESS [  1.711 s]
    [INFO] flink-dataset-fine-grained-recovery-test ........... SUCCESS [  1.954 s]
    [INFO] flink-datastream-allround-test ..................... SUCCESS [  6.846 s]
    [INFO] flink-batch-sql-test ............................... SUCCESS [  1.494 s]
    [INFO] flink-stream-sql-test .............................. SUCCESS [  1.221 s]
    [INFO] flink-bucketing-sink-test .......................... SUCCESS [  2.838 s]
    [INFO] flink-distributed-cache-via-blob ................... SUCCESS [  3.950 s]
    [INFO] flink-high-parallelism-iterations-test ............. SUCCESS [  9.000 s]
    [INFO] flink-stream-stateful-job-upgrade-test ............. SUCCESS [  4.041 s]
    [INFO] flink-queryable-state-test ......................... SUCCESS [  6.232 s]
    [INFO] flink-local-recovery-and-allocation-test ........... SUCCESS [  2.494 s]
    [INFO] flink-elasticsearch2-test .......................... SUCCESS [  2.876 s]
    [INFO] flink-elasticsearch5-test .......................... SUCCESS [  3.135 s]
    [INFO] flink-elasticsearch6-test .......................... SUCCESS [  7.638 s]
    [INFO] flink-quickstart ................................... SUCCESS [  2.395 s]
    [INFO] flink-quickstart-java .............................. SUCCESS [  4.188 s]
    [INFO] flink-quickstart-scala ............................. SUCCESS [  4.229 s]
    [INFO] flink-quickstart-test .............................. SUCCESS [  0.778 s]
    [INFO] flink-confluent-schema-registry .................... SUCCESS [  3.102 s]
    [INFO] flink-stream-state-ttl-test ........................ SUCCESS [  9.832 s]
    [INFO] flink-sql-client-test .............................. SUCCESS [  1.834 s]
    [INFO] flink-streaming-file-sink-test ..................... SUCCESS [  4.116 s]
    [INFO] flink-state-evolution-test ......................... SUCCESS [  5.834 s]
    [INFO] flink-rocksdb-state-memory-control-test ............ SUCCESS [  3.988 s]
    [INFO] flink-mesos ........................................ SUCCESS [ 40.768 s]
    [INFO] flink-kubernetes ................................... SUCCESS [ 15.522 s]
    [INFO] flink-yarn ......................................... SUCCESS [ 10.042 s]
    [INFO] flink-gelly ........................................ SUCCESS [ 13.021 s]
    [INFO] flink-gelly-scala .................................. SUCCESS [ 21.494 s]
    [INFO] flink-gelly-examples ............................... SUCCESS [ 18.450 s]
    [INFO] flink-metrics-dropwizard ........................... SUCCESS [  2.049 s]
    [INFO] flink-metrics-graphite ............................. SUCCESS [  0.828 s]
    [INFO] flink-metrics-influxdb ............................. SUCCESS [  3.054 s]
    [INFO] flink-metrics-prometheus ........................... SUCCESS [  2.329 s]
    [INFO] flink-metrics-statsd ............................... SUCCESS [  1.905 s]
    [INFO] flink-metrics-datadog .............................. SUCCESS [  1.171 s]
    [INFO] flink-metrics-slf4j ................................ SUCCESS [  1.896 s]
    [INFO] flink-cep-scala .................................... SUCCESS [ 18.199 s]
    [INFO] flink-table-uber ................................... SUCCESS [ 12.974 s]
    [INFO] flink-table-uber-blink ............................. SUCCESS [  5.012 s]
    [INFO] flink-sql-client ................................... SUCCESS [  5.989 s]
    [INFO] flink-state-processor-api .......................... SUCCESS [  2.643 s]
    [INFO] flink-python ....................................... SUCCESS [ 17.670 s]
    [INFO] flink-scala-shell .................................. SUCCESS [ 26.648 s]
    [INFO] flink-dist ......................................... SUCCESS [ 17.640 s]
    [INFO] flink-end-to-end-tests-common ...................... SUCCESS [  1.721 s]
    [INFO] flink-metrics-availability-test .................... SUCCESS [  1.180 s]
    [INFO] flink-metrics-reporter-prometheus-test ............. SUCCESS [  1.100 s]
    [INFO] flink-heavy-deployment-stress-test ................. SUCCESS [ 16.536 s]
    [INFO] flink-connector-gcp-pubsub-emulator-tests .......... SUCCESS [  9.727 s]
    [INFO] flink-streaming-kafka-test-base .................... SUCCESS [  4.911 s]
    [INFO] flink-streaming-kafka-test ......................... SUCCESS [  9.137 s]
    [INFO] flink-streaming-kafka011-test ...................... SUCCESS [  9.769 s]
    [INFO] flink-streaming-kafka010-test ...................... SUCCESS [ 10.255 s]
    [INFO] flink-plugins-test ................................. SUCCESS [  1.490 s]
    [INFO] dummy-fs ........................................... SUCCESS [  1.077 s]
    [INFO] another-dummy-fs ................................... SUCCESS [  1.101 s]
    [INFO] flink-tpch-test .................................... SUCCESS [  4.262 s]
    [INFO] flink-streaming-kinesis-test ....................... SUCCESS [ 20.838 s]
    [INFO] flink-elasticsearch7-test .......................... SUCCESS [  8.038 s]
    [INFO] flink-end-to-end-tests-common-kafka ................ SUCCESS [  2.841 s]
    [INFO] flink-tpcds-test ................................... SUCCESS [  2.483 s]
    [INFO] flink-statebackend-heap-spillable .................. SUCCESS [  2.171 s]
    [INFO] flink-contrib ...................................... SUCCESS [  1.579 s]
    [INFO] flink-connector-wikiedits .......................... SUCCESS [  5.509 s]
    [INFO] flink-yarn-tests ................................... SUCCESS [  7.158 s]
    [INFO] flink-fs-tests ..................................... SUCCESS [  7.135 s]
    [INFO] flink-docs ......................................... SUCCESS [  4.438 s]
    [INFO] flink-ml-parent .................................... SUCCESS [  1.599 s]
    [INFO] flink-ml-api ....................................... SUCCESS [  2.633 s]
    [INFO] flink-ml-lib ....................................... SUCCESS [ 10.818 s]
    [INFO] flink-walkthroughs ................................. SUCCESS [  2.391 s]
    [INFO] flink-walkthrough-common ........................... SUCCESS [  8.615 s]
    [INFO] flink-walkthrough-table-java ....................... SUCCESS [  4.231 s]
    [INFO] flink-walkthrough-table-scala ...................... SUCCESS [  4.092 s]
    [INFO] flink-walkthrough-datastream-java .................. SUCCESS [  4.293 s]
    [INFO] flink-walkthrough-datastream-scala ................. SUCCESS [  4.287 s]
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 09:45 min (Wall Clock)
    [INFO] Finished at: 2021-08-15T16:12:21+08:00
    [INFO] Final Memory: 589M/3489M
    [INFO] ------------------------------------------------------------------------08:00
    [INFO] Final Memory: 589M/3489M
    [INFO] ------------------------------------------------------------------------
    ``
    
    

step6:查看编译结果:

编译成功后,编译出完整的flink-binary,在源码目录flink-dist/target/中:

在这里插入图片描述

step7:启动本地模式测试

在这里插入图片描述

  • 运行wordcount测试
  • 去webUI查看任务结果
  • 1
    点赞
  • 5
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 3
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

海若[MATRIX]

鼓励将是我最大的动力!

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值