1. 环境准备
1.1. 处理Linux环境
a) 确保各主机拥有root权限
主要是为了避免安装过程中的权限问题
b) 关闭防火墙
[root@abc] systemctl stop firewalld
[root@abc] systemctl disable firewalld
c) 确保如下端口是开放的
https://www.cloudera.com/documentation/enterprise/6/6.1/topics/cm_ig_ports.html
d) 关闭SELinux
[root@abc] getenforce
如果输出为Disabled,则不需要修改配置,如果为enforcing,则需要做如下配置
[root@abc] vim /etc/selinux/config
#-----------------------------------
SELINUX=disabled
#-----------------------------------
:wq
[root@abc] setenforce 0
e) 调整交换内存阀值
[root@abc] sysctl vm.swappiness=10 && echo "vm.swappiness=10" >> /etc/sysctl.conf
f) 关闭大内存页配置
[root@abc] echo never > /sys/kernel/mm/transparent_hugepage/defrag && echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@abc] chmod +x /etc/rc.d/rc.local
[root@abc] echo "echo never > /sys/kernel/mm/transparent_hugepage/defrag" >> /etc/rc.local
[root@abc] echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.local
1.2. 配置HOSTNAME
主机规划:
IP |
主机名 |
配置 |
服务 |
10.6.222.85 |
txznwltesthadoop1 |
8c16g |
ClouderaManagerServer、ClouderaManagerAgent、ClouderaManagerService、HistoryServer(Spark)、JobHistoryServer(YARN)、HDFSBalancer |
10.6.222.86 |
txznwltesthadoop2 |
8c16g |
ClouderaManagerAgent、NameNode、ResourceManager、HMaster |
10.6.222.87 |
txznwltesthadoop3 |
24c48g |
ClouderaManagerAgent、DataName、NodeManager、HRegionServer、SparkGateway |
10.6.222.88 |
txznwltesthadoop4 |
24c48g |
ClouderaManagerAgent、DataName、NodeManager、HRegionServer、SparkGateway |
10.6.222.89 |
txznwltesthadoop5 |
24c48g |
ClouderaManagerAgent、DataName、NodeManager、HRegionServer、SparkGateway |
10.6.222.90 |
txznwltesthadoop6 |
8c16g |
ClouderaManagerAgent、ZooKeeper、KafkaBroker |
10.6.222.91 |
txznwltesthadoop7 |
8c16g |
ClouderaManagerAgent、ZooKeeper、KafkaBroker |
10.6.222.92 |
txznwltesthadoop8 |
8c16g |
ClouderaManagerAgent、ZooKeeper、KafkaBroker |
[root@abc] vim /etc/hostname
#----------------------------------------
txznwltesthadoop1 # 对应的主机设置对应的主机名
#----------------------------------------
:wq
[root@abc] vim /etc/sysconfig/network
#----------------------------------------
HOSTNAME=txznwltesthadoop1 # 对应的主机设置对应的主机名
#----------------------------------------
:wq
[root@abc] hostname txznwltesthadoop1 # 对应的主机设置对应的主机名
[root@abc] vim /etc/hosts
#----------------------------------------
10.6.222.85 txznwltesthadoop1
10.6.222.86 txznwltesthadoop2
10.6.222.87 txznwltesthadoop3
10.6.222.88 txznwltesthadoop4
10.6.222.89 txznwltesthadoop5
10.6.222.90 txznwltesthadoop6
10.6.222.91 txznwltesthadoop7
10.6.222.92 txznwltesthadoop8
#----------------------------------------
:wq
注意:这里必须设置,并且主机名不能有下划线
1.3. 配置SSH免密
使用ssh-copy-id进行免密配置
[root@abc] ssh-keygen -t rsa
# 不需要设置密码,直接全部回车
[root@abc] ss