Hadoop学习：Linux集群搭建

在写此文之前，郑重告诉大家，别轻易尝试rm -rf /*#这个命令。

本文辅看视频：点链接即可>>> 🔗

Hadoop学习：Linux集群搭建

一、Linux虚拟机环境搭建

先建立好一台虚拟机，命名好

配置Linux的网络环境:

1. 编辑虚拟机
   
   

2. 进入虚拟机，在/etc目录下配置虚拟机IP

centos7目录为:
vi /etc/sysconfig/network-scripts/ifcfg-ens33

centos6目录为：
vi /etc/sysconfig/network-scripts/ifcfg-eth0

摁 i 进入编辑模式

3. 编辑内容，编辑点我用#注释出来了

[root@hadoop-1 /]# vi /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"

BOOTPROTO="static"    #原来为dfhc,改成静态，避免开机自动动态分布

DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="ens33"
UUID="92af196d-aeb1-4619-85dd-33596e57bfc3"
DEVICE="ens33"

ONBOOT="yes"    #是no的改成yes

    #以下全是编辑内容，根据自己所需

IPADDR="192.168.2.129"
GATEWAY="192.168.2.1"
BROADCAST="192.168.2.255"
DNS1=114.114.114.114
DNS2=8.8.8.8

4. 更改网名

命令为：

vi /etc/sysconfig/network

编辑内容为：

NETWORKING=yes
HOSTNAME=你想命名的主机名

5. 停止网卡服务

service NetworkManager stop

6. 重启网卡

/etc/init.d/network restart

7. 关闭自动重启

chkconfig NetworkManager off

8. 重启网络

service network restart

9. 防火墙的关闭

关闭防火墙

systemctl stop firewalld.service

关闭开机自启

systemctl disable firewalld.service

注意：centos7和6命令是不一样的。
centos6

service iptables stop

shkconfig iptables off

自此，Linux的一台虚拟机配置完成，其他虚拟机只需要将该虚拟机克隆，修改IP和物理地址，也可以再配三台。

二、secureCRT的集群操作

1. secureCRT

先下载

利用破解包破解后，设置密码，并登录。

SecureCRT_7.3注册机激活步骤如下：

1）准备工作

安装好SecureCRT软件，

下载并得到该注册机。

2）保持SecureCRT软件关闭（运行的话会提示你正在运行的，关闭就好）。

3）将注册机拷贝到你的CRT软件的安装的目录下；

4）点击【Patch】按钮，会让你选择文件
，选择SecureCRT.exe可执行文件，还会再提示一次选择一个help什么文件。总之，确定后都会提示成功信息。

5）然后打开CRT软件，按照相应的提示输入注册机上显示的信息（当然重新生成也是可以的）。注意，默认打开CRT时候会是一个输入key的页面，什么都不输入，点击下一步，选择输入自己的相关信息。就会看到分别让你输入名称、序列号等相关日期了。完成结束。看看自己的是否有激活成功。

连接虚拟机：

并登录即可。

2. 设置SSH免密登录

先生成私钥和公钥，私钥发给自己，公钥发其他。

命令1：

ssh-keygen -t rsa

生成密钥结果：

[root@hadoop-3 /]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:3uHospoo8z0WQ+o4zSTz/gsCEAbmqcXlQynwEn/ZKWs root@hadoop-3
The key's randomart image is:
+---[RSA 2048]----+
|*+  o.           |
|+*o+.o .         |
|oo=.* o          |
|oo ..+           |
|o  oE   S .      |
|+ o.o  . + .     |
|.X.  o  o o      |
|=.=+o...         |
| *+o*+.o.        |
+----[SHA256]-----+

命令2：

私钥发本机：

ssh-sopy-id localhost

命令3：

公钥发要控制的机器：

ssh-copy-id hadoop-2

这样就可以通信了，结果如下：

[root@hadoop-1 /]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:iZXvsuC4sgNuPGJUqrYx+fy/YZRPfQQsOw94X1JizoA root@hadoop-1
The key's randomart image is:
+---[RSA 2048]----+
|        . ..     |
|       E + +..   |
|        + B o.   |
|   .   +.B.+..   |
|  o   .oS.*.o.   |
|.o.   . o. o.    |
|==    .o...      |
|oBB  o...o       |
|=.+*+oooo        |
+----[SHA256]-----+
[root@hadoop-1 /]# ssh-copy-id localhost
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:ISWgQqsAxnz/J02aBTwOOGL2lYtb5gkInT/i9SjmSOY.
ECDSA key fingerprint is MD5:79:37:35:7f:84:f3:48:a2:9a:7e:34:c5:88:aa:ef:fe.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@localhost's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'localhost'"
and check to make sure that only the key(s) you wanted were added.

[root@hadoop-1 /]# ssh-copy-id Hadoop-2
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@hadoop-2's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'Hadoop-2'"
and check to make sure that only the key(s) you wanted were added.

[root@hadoop-1 /]# ssh-copy-id Hadoop-3
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'hadoop-3 (192.168.2.128)' can't be established.
ECDSA key fingerprint is SHA256:5StSMlwvq8m9jnzX7vGpGndDUN/j+UxEBdZJ+rcvoKU.
ECDSA key fingerprint is MD5:8b:ab:1a:c3:fc:dd:4e:31:ed:61:3b:68:7f:7f:9a:bd.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@hadoop-3's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'Hadoop-3'"
and check to make sure that only the key(s) you wanted were added.


     #这里就可以免密登录了#

[root@hadoop-1 /]# ssh Hadoop-3
Last login: Tue Jul  7 00:51:36 2020 from 192.168.2.2
[root@hadoop-3 ~]# exit