Hadoop集群部署

版本
JRE：jdk-8u45-linux-x64.gz
hadoop：hadoop-2.6.0-cdh5.15.1.tar.gz
zookeeper：zookeeper-3.4.6.tar.gz

基本操作
1、创建hadoop用户并切入

[root@ruozedata001 ~]# useradd hadoop
[root@ruozedata001 ~]# su - hadoop

2、创建目录

[hadoop@ruozedata001 ~]$ mkdir app software source data lib script tmp maven_repos
[hadoop@ruozedata001 ~]$ ll
total 32
drwxrwxr-x 2 hadoop hadoop 4096 Aug 21 14:36 app
drwxrwxr-x 2 hadoop hadoop 4096 Aug 21 14:36 data
drwxrwxr-x 2 hadoop hadoop 4096 Aug 21 14:36 lib
drwxrwxr-x 2 hadoop hadoop 4096 Aug 21 14:36 maven_repos
drwxrwxr-x 2 hadoop hadoop 4096 Aug 21 14:36 script
drwxrwxr-x 2 hadoop hadoop 4096 Aug 21 14:36 software
drwxrwxr-x 2 hadoop hadoop 4096 Aug 21 14:36 source
drwxrwxr-x 2 hadoop hadoop 4096 Aug 21 14:36 tmp

3、which查找是否有rz上传命令，没有则创建

[hadoop@ruozedata001 software]$ which rz
/usr/bin/which: no rz in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hadoop/.local/bin:/home/hadoop/bin)
[hadoop@ruozedata001 software]$ exit
logout
[root@ruozedata001 ~]# yum install -y lrzsz
Loaded plugins: fastestmirror
base                                                        | 3.6 kB  00:00:00     
epel                                                        | 5.3 kB  00:00:00     
extras                                                      | 3.4 kB  00:00:00     
updates                                                     | 3.4 kB  00:00:00     
(1/7): epel/x86_64/group_gz                                 |  88 kB  00:00:00     
(2/7): base/7/x86_64/group_gz                               | 166 kB  00:00:00     
(3/7): extras/7/x86_64/primary_db                           | 215 kB  00:00:00     
(4/7): epel/x86_64/primary_db                               | 6.8 MB  00:00:00     
(5/7): epel/x86_64/updateinfo                               | 999 kB  00:00:00     
(6/7): base/7/x86_64/primary_db                             | 6.0 MB  00:00:00     
(7/7): updates/7/x86_64/primary_db                          | 7.4 MB  00:00:00     
Determining fastest mirrors
Resolving Dependencies
--> Running transaction check
---> Package lrzsz.x86_64 0:0.12.20-36.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===================================================================================
 Package         Arch             Version                     Repository      Size
===================================================================================
Installing:
 lrzsz           x86_64           0.12.20-36.el7              base            78 k

Transaction Summary
===================================================================================
Install  1 Package

Total download size: 78 k
Installed size: 181 k
Downloading packages:
lrzsz-0.12.20-36.el7.x86_64.rpm                             |  78 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : lrzsz-0.12.20-36.el7.x86_64                                     1/1 
  Verifying  : lrzsz-0.12.20-36.el7.x86_64                                     1/1 

Installed:
  lrzsz.x86_64 0:0.12.20-36.el7                                                    

Complete!

4、rz上传

[root@ruozedata001 ~]# su - hadoop
Last login: Wed Aug 21 14:34:26 CST 2019 on pts/0
[hadoop@ruozedata001 ~]$ cd software/
[hadoop@ruozedata001 software]$ ll
total 0
[hadoop@ruozedata001 software]$ rz   
[hadoop@ruozedata001 software]$ ll
total 433188
-rw-r--r-- 1 hadoop hadoop 252606214 Aug 21 14:46 hadoop-2.6.0-cdh5.15.1.tar.gz
-rw-r--r-- 1 hadoop hadoop 173271626 Jun 12 02:00 jdk-8u45-linux-x64.gz
-rw-r--r-- 1 hadoop hadoop  17699306 Aug 21 14:44 zookeeper-3.4.6.tar.gz

环境准备
1、配置每台机器的etc/host文件，etc/host文件里必须配置内网ip，千万不能是外网ip
#此时可以切换到三台机器一块做，退到root用户，vi进入

[root@ruozedata003 ~]# vi /etc/hosts
172.18.129.90  ruozedata001
172.18.129.88  ruozedate002
172.18.129.89  ruozedata003

#hosts文件的第一行第二行不要删掉

hosts文件的第一行第二行不要删掉
 
 [root@ruozedata001 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6


172.18.129.90  ruozedata001
172.18.129.88  ruozedata002
172.18.129.89  ruozedata003

2、设置hadoop用户的互相信任关系
1）3台机器执行

[hadoop@ruozedata001 ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 
Created directory '/home/hadoop/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
5d:aa:1e:a4:60:de:3e:79:25:5e:6f:3f:d1:2b:96:a1 hadoop@ruozedata001
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|            .    |
|         . o     |
|    o   S