linux安装hadoop集群（1）-准备工作

1 准备三台服务器

192.168.199.210 hpmaster

192.168.199.211 hpslave1

192.168.199.212 hpslave2

2 配置hosts

三台主机都配置hosts

[root@localhost ~]# vim /etc/hosts

192.168.199.210 hpmaster

192.168.199.211 hpslave1

192.168.199.212 hpslave2

3 测试ping

保证三台主机互相ping通

[root@hpmaster 桌面]# ping hpslave1
PING hpslave1 (192.168.199.211) 56(84) bytes of data.
64 bytes from hpslave1 (192.168.199.211): icmp_seq=1 ttl=64 time=0.384 ms
64 bytes from hpslave1 (192.168.199.211): icmp_seq=2 ttl=64 time=0.563 ms
64 bytes from hpslave1 (192.168.199.211): icmp_seq=3 ttl=64 time=2.02 ms

4 安装jdk

三台服务器配置java环境变量

[root@hpmaster ~]# vim /etc/profile

export JAVA_HOME=/usr/local/jdk1.8.0_181

export JRE_HOME=/usr/local/jdk1.8.0_181/jre

export CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib

export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

5 添加hadoop用户账户

三台服务器添加hadoop用户账户

[root@localhost ~]# useradd hadoop

[root@localhost ~]# passwd hadoop

密码 hadoop

6 配置hadoop用户组

三台服务器hadoop用户添加到hadoop用户组

[root@localhost ~]# usermod -a -G hadoop hadoop

[root@localhost ~]# cat /etc/group

7 设置hadoop用户权限

三台服务器设置hadoop用户具有root权限

[root@localhost ~]# visudo

## Allow root to run any commands anywhere

root    ALL=(ALL)       ALL

hadoop  ALL=(ALL)       ALL

[root@localhost ~]# cat /etc/sudoers

8 永久关闭防火墙

三台服务器，修改防火墙配置文件，修改完毕后重新启动linux系统

[root@localhost ~]# vi /etc/selinux/config

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

#     enforcing - SELinux security policy is enforced.

#     permissive - SELinux prints warnings instead of enforcing.

#     disabled - No SELinux policy is loaded.

SELINUX=disabled

9 ssh免密码登录

在hpmaster节点 ，root用户下输入 ssh-keygen -t rsa 一路回车

[root@hpmaster ~]# ssh-keygen -t rsa

+---[RSA 2048]----+

|=o+   .          |

|+Bo*.=           |

|*+/.O.o          |

|+X @ +.+    .    |

|  = o ..S  o o   |

|      o   E . +  |

|     . . .   o . |

|        o .   .  |

|         o.o.    |

+----[SHA256]-----+

秘钥生成后在~/.ssh/目录下，有两个文件id_rsa(私钥)和id_rsa.pub（公钥）

[root@hpmaster ~]# cd ~/.ssh

[root@hpmaster .ssh]# ls

id_rsa  id_rsa.pub

将公钥复制到authorized_keys

[root@hpmaster .ssh]# cat  ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

[root@hpmaster .ssh]# ls

authorized_keys  id_rsa  id_rsa.pub

并赋予authorized_keys 600权限

[root@hpmaster .ssh]# chmod 600 ~/.ssh/authorized_keys

另两台服务器进行同样的操作完成以上步骤

10 authoized_keys合并

将三台服务器的authoized_keys互相拼接在一个文件中，最终三者的authoized_keys内容一致

Master authoized_keys 远程拷贝到其他服务的authoized_keys中

[root@hpmaster ~]# ssh-copy-id -i .ssh/id_rsa.pub  root@192.168.199.211

[root@hpmaster ~]# ssh-copy-id -i .ssh/id_rsa.pub  root@192.168.199.212

Slave1 authoized_keys 远程拷贝到其他服务的authoized_keys中

[root@hpslave1~]# ssh-copy-id -i .ssh/id_rsa.pub  root@192.168.199.210

[root@hpslave1~]# ssh-copy-id -i .ssh/id_rsa.pub  root@192.168.199.212

Slave2 authoized_keys 远程拷贝到其他服务的authoized_keys中

[root@hpslave2~]# ssh-copy-id -i .ssh/id_rsa.pub  root@192.168.199.210

[root@hpslave2~]# ssh-copy-id -i .ssh/id_rsa.pub  root@192.168.199.211

[root@hpmaster .ssh]# cat authorized_keys

[root@hpslave1 .ssh]# cat authorized_keys

[root@hpslave2 .ssh]# cat authorized_keys  

最终三台服务器的authoized_keys内容一致

11 检查是否免密登录

第一次登录会有提示

[root@hpslave1 ~]# ssh hpslave2
The authenticity of host 'hpslave2 (192.168.199.212)' can't be established.
ECDSA key fingerprint is SHA256:e8ynknd2V+RguccraeIjg6e7gdDni2wRDtdHKLBISfY.
ECDSA key fingerprint is MD5:68:13:8f:e1:34:35:26:28:4e:6a:fe:b8:a7:46:35:8c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hpslave2,192.168.199.212' (ECDSA) to the list of known hosts.
Last login: Tue Jan 15 00:19:07 2019 from hpmaster
vm.max_map_count = 262144

再次登录

[root@hpmaster ~]# ssh hpslave2
Last login: Tue Jan 15 00:33:52 2019 from hpmaster
vm.max_map_count = 262144

三台互相登录访问下