CDH5.16.1集群企业真正离线部署(三台阿里云主机按量付费)

一.准备工作

1.集群规划
 

节点	MySQL部署组件	Parcel文件离线源	CM服务进程	大数据组件
hadoop001	MySQL	Parcel	Activity Monitor	NN RM DN NM
hadoop002			Alert Publisher Event Server	DN NM
hadoop003			Alert Publisher Event Server	DN NM

2.下载源
CM：
http://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz
Parcel：
http://archive.cloudera.com/cdh5/parcels/5.16.1/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel
http://archive.cloudera.com/cdh5/parcels/5.16.1/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.16.1/manifest.json
JDK：
https://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html
MySQL
https://dev.mysql.com/downloads/mysql/5.7.html#downloads
MySQL jdbc jar
http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar
下载完成后要重命名去掉版本号，
mv mysql-connector-java-5.1.47.jar mysql-connector-java.jar

二.集群节点初始化

1.阿里云上海海区购买3台，按量量付费虚拟机
CentOS7.2操作系统，2核8G最低配置
2.当前笔记本或台式机配置hosts文件
MAC: /etc/hosts
Window: C:\windows\system32\drivers\etc\hosts

172.17.79.118 hadoop001
172.17.79.117 hadoop002
172.17.79.116 hadoop003

3.设置所有节点的hosts文件
vim /etc/hosts

172.17.79.118 hadoop001
172.17.79.117 hadoop002
172.17.79.116 hadoop003

4.关闭所有节点的防火墙及清空规则
centos关闭防火墙

systemctl stop firewalld
systemctl disable firewalld
iptables -F
iptables -L

一般阿里云的防火墙都是关闭的，但是阿里云服务器上有一个Web的防火墙，要打开7180的端口




5.关闭所有节点的selinux
[root@hadoop001 ~]# vim /etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled
设置后需要重启才能生效
[root@hadoop001 ~]# reboot
6.查看设置所有节点的时区一致及时钟同步
6.1 时区

[root@hadoop001 ~]# date
Sat Oct  5 19:22:21 CST 2019
[root@hadoop001 ~]# timedatectl
      Local time: Sat 2019-10-05 19:21:55 CST
  Universal time: Sat 2019-10-05 11:21:55 UTC
        RTC time: Sat 2019-10-05 19:21:54
       Time zone: Asia/Shanghai (CST, +0800)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: yes
      DST active: n/a

Warning: The system is configured to read the RTC time in the local time zone.
         This mode can not be fully supported. It will create various problems
         with time zone changes and daylight saving time adjustments. The RTC
         time is never updated, it relies on external facilities to maintain it.
         If at all possible, use RTC in UTC by calling
         'timedatectl set-local-rtc 0'.
[root@hadoop001 ~]# 

如果时区不一致，使用如下命令

[root@hadoop001 ~]# timedatectl set-timezone Asia/Shanghai

6.2 时间
为所有节点安装ntp

[root@hadoop001 ~]# yum install -y ntp

选取hadoop001为ntp的主节点
[root@hadoop001 ~]# vi /etc/ntp.conf

server 0.asia.pool.ntp.