Flink1.11.2集成CDH6.0.1
1. 集群规划
普通分布式集群
一个简单的表格是这么创建的:
节点名称 | master | slave1 | slave2 |
---|---|---|---|
JobManager | √ | ||
TaskManager | √ | √ | √ |
2. 版本选择
- CentOS 7.5
- CDH 6.0.1
- Scala 2.11
- Flink 1.11.2
3. Flink集成CDH编译
3.1 准备maven环境
- 获取maven安装包
wget https://mirror.bit.edu.cn/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
- 解压文件
tar -zxvf apache-maven-3.6.3-bin.tar.gz
- 配置环境变量 /etc/profile
export MAVEN_HOME=/usr/local/apache-maven-3.6.3
export PATH=$MAVEN_HOME/bin:$PATH
- 使环境变量生效
source /etc/profile
- 修改setting.xml:
vim apache-maven-3.6.3/conf/settings.xml
添加如下配置
<mirrors>
<mirror>
<id>alimaven</id>
<mirrorOf>central</mirrorOf>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/repositories/central/</url>
</mirror>
<mirror>
<id>alimaven</id>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/groups/public/</url>
<mirrorOf>central</mirrorOf>
</mirror>
<mirror>
<id>central</id>
<name>Maven Repository Switchboard</name>
<url>http://repo1.maven.org/maven2/</url>
<mirrorOf>central</mirrorOf>
</mirror>
<mirror>
<id>repo2</id>
<mirrorOf>central</mirrorOf>
<name>Human Readable Name for this Mirror.</name>
<url>http://repo2.maven.org/maven2/</url>
</mirror>
<mirror>
<id>ibiblio</id>
<mirrorOf>central</mirrorOf>
<name>Human Readable Name for this Mirror.</name>
<url>http://mirrors.ibiblio.org/pub/mirrors/maven2/</url>
</mirror>