目录
启动
简单启动
下载所需内容
[doudou@hadoop102 flink]$ ll
total 510348
-rw-r--r-- 1 doudou doudou 149499945 10月 9 04:01 elasticsearch-6.8.5.tar.gz
-rw-r--r-- 1 doudou doudou 305107189 10月 9 04:01 flink-1.13.0-bin-scala_2.12.tgz
-rw-r--r-- 1 doudou doudou 61914514 10月 9 04:01 kafka_2.11-2.1.0.tgz
-rw-r--r-- 1 doudou doudou 3597701 10月 9 04:00 mysql-connector-java-5.1.28.tar.gz
-rw-r--r-- 1 doudou doudou 2465302 10月 9 04:00 redis-6.2.5.tar.gz
启动进程
[doudou@hadoop102 module]$ cd flink-1.13.0/
[doudou@hadoop102 flink-1.13.0]$ bin/start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host hadoop102.
Starting taskexecutor daemon on host hadoop102.
[doudou@hadoop102 flink-1.13.0]$ jps
12072 Jps
11769 StandaloneSessionClusterEntrypoint
12044 TaskManagerRunner
修改conf的masters和workers
[doudou@hadoop102 flink-1.13.0]$ cat conf/workers
hadoop103
hadoop104
[doudou@hadoop102 flink-1.13.0]$ cat conf/masters
hadoop102:8081
集群启动
分发
[doudou@hadoop102 flink-1.13.0]$ xsync flink-1.13.0/
启动
[doudou@hadoop102 flink-1.13.0]$ ./bin/start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host hadoop102.
Starting taskexecutor daemon on host hadoop103.
Starting taskexecutor daemon on host hadoop104.
向集群提交作业
可以引入插件 maven-assembly-plugin 进行打包。在 FlinkTutorial 项目的 pom.xml 文件中添加打包插件的配置
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
向命令行提交
传jar包
开放hadoop102
在hadoop103上开启
[doudou@hadoop103 flink-1.13.0]$ ./bin/flink run -m hadoop102:8081 -c com.atguigu.wc.StreamWordCount -p 2 ./FinkTutorial-1.0-SNAPSHOT.jar
关闭释放空间
[doudou@hadoop103 flink-1.13.0]$ ./bin/flink cancel c2e89d6899b1300ecde1cad336165ea9
YARN模式
配置环境变量
[doudou@hadoop102 flink-1.13.0]$ sudo vim /etc/profile.d/my_env.sh
HADOOP_HOME=/opt/module/hadoop-2.7.5
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HADOOP_CLASSPATH=`hadoop classpath`
会话模式部署
执行脚本命令向 YARN 集群申请资源,开启一个 YARN 会话,启动 Flink 集群
bin/yarn-session.sh -nm test
提交作业
[doudou@hadoop102 flink-1.13.0]$ ./bin/flink run -c com.atguigu.wc.StreamWordCount ./FinkTutorial-1.0-SNAPSHOT.jar
单作业模式
执行命令提交作业
bin/flink run -d -t yarn-per-job -c com.atguigu.wc.StreamWordCount
FlinkTutorial-1.0-SNAPSHOT.jar