1.说明
目标
1)三台机器Centos7
机器 | JobManager | TaskManager | JouralNode | DataNode | NameNode | Zkfc | Zookeeper |
---|---|---|---|---|---|---|---|
192.168.11.131 | √ | √ | √ | √ | √ | √ | √ |
192.168.11.132 | - | √ | √ | √ | √ | √ | √ |
192.168.11.133 | √ | √ | √ | √ | - | - | √ |
即JobManager部署131、132两台服务器,而NameNode部署131、133(Zkfc同步部署)
2)实现自启动上述服务
前提
1)jdk已安装
2)zookeeper的安装、自启动请参考:https://blog.csdn.net/sndayYU/article/details/100537922、https://blog.csdn.net/sndayYU/article/details/114047145
具体操作
1.免密登录
参考:https://blog.csdn.net/sndayYU/article/details/115036682
用户flink,别名分别时node1、node2、node3
注意
同理131的root免密到132、133
2.相关目录
/flink/为安装flink、hadoop、zookeeper安装目录,/data/为其数据及日志目录
chown flink:flink /data
chown flink:flink /flink
chmod 755 /data
chmod 755 /flink
3.zookeeper
基本配置
zookeeper的安装、自启动请参考:https://blog.csdn.net/sndayYU/article/details/100537922、https://blog.csdn.net/sndayYU/article/details/114047145
- 安装目录/flink/zookeeper-3.4.14/
- data目录/data/zookeeper/data/
- log目录/data/zookeeper/log/
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/logs
server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888
自启动
(参考:https://blog.csdn.net/sndayYU/article/details/114047145):
systemctl start zookeeper
脚本/lib/systemd/system/zookeeper.service
[Unit]
Description=zookeeper
[Service]
Type=forking
Environment=ZOO_LOG_DIR=/data/zookeeper
Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64
ExecStart=/flink/zookeeper-3.4.14/bin/zkServer.sh start
ExecStop=/flink/zookeeper-3.4.14/bin/zkServer.sh stop
PrivateTmp=true
User=flink
Group=flink
[Install]
WantedBy=multi-user.target
4.hadoop
基本配置
解压,配置core-site.xml、hdfs-site.xml、slaves、删除masters,注意检查目录权限是否为flink。
传到另外2台。
core-site.xml
<property>
<name>fs.defaultFS</name>
<!-- 要与hdfs-site.xml的dfs.nameservices 保持一直 -->
<value>hdfs://hadoopha</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<!-- uat配置 file:/data/hadoop/tmp,不知道会不会有问题 -->
<value>/data/hadoop/tmp</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>node1:2181,node2:2181,node3:2181</value>
</property>
<property>
<name>ha.zookeeper.session-timeout.ms</name>
<value>150000</value>
</property>
hdfs-site.xml
<property>
<!-- 不需要file://会报错 -->
<name>dfs.namenode.name.dir</name>
<value>file:///data/hadoop/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///data/hadoop/data</value>
</property>
<!-- 采用默认值3 -->
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<!-- core-site.xml有使用到 -->
<property>
<name>dfs.nameservices</name>
<value>hadoopha</value>
</property>
<property>
<!-- dfs.ha.namenodes.EXAMPLENAMESERVICE -->
<name>dfs.ha.namenodes.hadoopha</name>
<value>nn1,nn2</value>
</property>
<property>
<!-- <name>dfs.namenode.rpc-address</name> -->
<name>dfs.namenode.rpc-address.hadoopha.nn1</name>
<value>node1:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.hadoopha.nn1</name>
<value>node1:50070</value>
</property>
<property>
<!-- <name>dfs.namenode.rpc-address</name> -->
<name>dfs.namenode.rpc-address.hadoopha.nn2</name>
<value>node2:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.hadoopha.nn2</name>
<value>node2:50070</value>
</property>
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://node1:8485;node2:8485;node3:8485/hadoopha</value>
</property>
<!-- 指定JournalNode在本地磁盘存放数据的位置 -->
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/data/hadoop/journal</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.hadoopha</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>shell(/bin/true)</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.connect-timeout</name>
<value>30000</value>
</property>
[root@node1 ~]# cat /etc/profile.d/my.sh
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64
export CALSSPATH=$JAVA_HOME/lib/*.*
export PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/flink/hadoop-2.6.5
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
初始化
注意:若是不能一次性搞定,需要清空/data/hadoop目录,及zookeeper的相关数据
1)my.sh环境变量,hadoop version验证下,可以重启下服务器
2)3台启动zookeeper,及 切换flink 执行journalnode
/flink/hadoop-2.6.5/sbin/hadoop-daemon.sh start journalnode
3)node1:
hdfs namenode -format
/flink/hadoop-2.6.5/sbin/hadoop-daemon.sh start namenode
4) node2:
hdfs namenode -bootstrapStandby
5) node1:
hdfs zkfc -formatZK
/flink/hadoop-2.6.5/sbin/start-dfs.sh
6) 最后访问node1:50070、node2:50070
自启动
略
5.flink
基本配置
flink用户解压,配置flink-conf.yaml、slaves、masters,
hadoop的flink-shaded-hadoop-2-uber-2.6.5-7.0.jar放入flink的lib目录
传到另外2台服务器
./flink-1.8.0/log目录一定要有,否则启动会报错
flink-conf.yaml
high-availability: zookeeper
high-availability.storageDir: hdfs:///hadoopha/ha/
high-availability.zookeeper.quorum: node1:2181,node2:2181,node3:2181
state.backend: filesystem
state.checkpoints.dir: hdfs:///hadoopha/flink-checkpoints
slaves
node1
node2
node3
masters
node1:8081
node3:8081
启动
[flink@node1 bin]$ ./start-cluster.sh
Starting HA cluster with 2 masters.
Starting standalonesession daemon on host node1.
Starting standalonesession daemon on host node3.
Starting taskexecutor daemon on host node1.
Starting taskexecutor daemon on host node2.
Starting taskexecutor daemon on host node3.
[flink@node1 bin]$ jps -----验证是否启动正确
访问:http://node1:8081/、http://node3:8081/
自启动
[root@node1 ~]# cat /lib/systemd/system/flink-jobmanager.service
[Unit]
Description=flink-jobmanager
[Service]
Type=forking
Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64
Environment=HADOOP_CONF_DIR=/flink/hadoop-2.6.5/etc/hadoop
ExecStart=/flink/flink-1.8.0/bin/jobmanager.sh start
ExecStop=/flink/flink-1.8.0/bin/jobmanager.sh stop
PrivateTmp=true
User=flink
Group=flink
[Install]
WantedBy=multi-user.target
[root@node1 ~]# cat /lib/systemd/system/flink-taskmanager.service
[Unit]
Description=flink-taskmanager
[Service]
Type=forking
Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64
Environment=HADOOP_CONF_DIR=/flink/hadoop-2.6.5/etc/hadoop
ExecStart=/flink/flink-1.8.0/bin/taskmanager.sh start
ExecStop=/flink/flink-1.8.0/bin/taskmanager.sh stop
PrivateTmp=true
User=flink
Group=flink
[Install]
WantedBy=multi-user.target
// 3.scp传到node2、node3
// 4.systemctl daemon-reload
node1、node3: systemctl start flink-jobmanager
node1、node2、node3: systemctl start flink-taskmanager
及systemctl enable xxxx
最终结果
自启动都配置好了后,重启3台电脑,可以发现都正确了,可以上传任务执行jar包
常见错误
- /etc/sudoers里面的权限:flink ALL=(ALL) NOPASSWD: ALL
- nn1和nn2需要配置正确
- zookeeper当测试时不用集群也是可以的,上面的集群配置相应需要做修改
- slaves里面配置的hosts就是datanode节点,start-dfs.sh启动时起作用