集群配置
准备工作
1、 Iptables
#chkconfig iptables –list
#chkconfig iptables –level 3456off
#service iptables stop
2、 Initab
#vim /etc/inittab
id:3
3、 Ntpd
#chkconfig ntpd –list
#chkconfig ntpd –level 23456off
#service iptables start
4、 Sudo hadoop(添加hadoop用户,赋于sudo权限)
#useradd hadoop
#passwd hadoop
#vim /etc/sudoers
加入hadoop ALL=(ALL) ALL
5、 Selinux disabled
#vim /etc/selinux/config
设置SELINUX=disabled
6、 Host
#vim /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=spark04
#hostname spark04
#vim /etc/hosts 主机名最好有意义的域名,例如:www.spark01.cn
10.36.0.159database
10.36.0.158spark03
10.36.0.157spark02
10.36.0.30spark01
10.36.0.201spark04
10.36.0.202spark05
7、 ssh
#ssh-keygen
#vim authorized_keys
把每台机子上的id_rsa.pub内容拷贝到authorized_keys
Scp到其它机器上
注意:配置完上述以后最好重启,配置才能生效。网络配置请符合基本配置要求
版本
名称 | 版本号 |
Hadoop | hadoop-2.6.0 |
Zookeeper | zookeeper-3.4.6 |
Jdk | jdk-7u75-linux-x64 |
主机(请按实际情况填写)
主机名 | IP |
database | 10.36.0.159 |
Spark02 | 10.36.0.157 |
Spark03 | 10.36.0.158 |
Spark04 | 10.36.0.201 |
Spark05 | 10.36.0.202 |
角色配置(请按实际情况填写)
database(159) | spark02(157) | spark03(158) | spark04(201) | spark05(202) | |
Namenode | Namenode | Namenode |
|
|
|
Datanode |
|
| Datanode | Datanode | Datanode |
Zookeeper | Zookeeper | Zookeeper | Zookeeper | Zookeeper | Zookeeper |
Journalnode | Journalnode | Journalnode | Journalnode |
|
|
zkfc | zkfc | zkfc |
|
|
|
Resourcemanager | Resourcemanager | Resourcemanager |
|
|
|
Nodemanager |
|
| Nodemanager | Nodemanager | Nodemanager |
Ha角色(请按实际情况填写)
角色名 | 主机名 |
NameNode | Database , spark02 |
ResourceManager | Database , spark02 |
访问地址(请按实际情况填写)
角色 | 访问地址 |
Namenode | http://database:50070 || http://spark02:50070 |
ResourceManager | http://database:8088 || http://spark02:8088 |
注:本地需要配置host(C:\Windows\System32\drivers\etc)