一、集群规划
hadoop1 | hadoop2 | hadoop3 | |
NameNode | √ | ||
SecondaryNameNode | √ | ||
DataNode | √ | √ | √ |
ResourceManager | √ | ||
DataManager | √ | √ | √ |
Hive | √ | √ | √ |
HiveServer2 | √ |
框架 | HDFS | YARN |
Hadoop01 | NameNode DataNode | ResourceManager DataManager |
Hadoop02 | SecondaryNameNode DataNode | DataManager |
Hadoop03 | DataNode | DataManager |
一、安装Hadoop:单节点配置
### --- 登录Hadoop01节点;进入/opt/yanqi/software,解压安装文件到/opt/yanqi/servers
[root@hadoop01 ~]# cd /opt/yanqi/software/
[root@hadoop01 software]# tar -zxvf hadoop-2.9.2.tar.gz -C /opt/yanqi/servers
### --- 查看是否解压成功
[root@hadoop01 ~]# ll /opt/yanqi/servers/hadoop-2.9.2
### --- 添加Hadoop到环境变量 vim /etc/profile
~~~ # 所有节点配置环境变量
[root@hadoop01 ~]# vim /etc/profile
##HADOOP_HOME
export HADOOP_HOME=/opt/yanqi/servers/hadoop-2.9.2
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
### --- 使环境变量生效
~~~ # 所有节点配置环境变量
[root@hadoop01 ~]# source /etc/profile
### --- 验证hadoop
[root@hadoop01 ~]# hadoop version
Hadoop 2.9.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r 826afbeae31ca687bc2f8471dc841b66ed2c6704
Compiled by ajisaka on 2018-11-13T12:42Z
Compiled with protoc 2.5.0
From source with checksum 3a9939967262218aa556c684d107985
This command was run using /opt/yanqi/servers/hadoop-2.9.2/share/hadoop/common/hadoop-common-2.9.2.jar
### --- hadoop目录结构
[root@hadoop01 ~]# ll /opt/yanqi/servers/hadoop-2.9.2/
total 128
drwxr-xr-x 2 501 dialout 194 Nov 13 2018 bin
drwxr-xr-x 3 501 dialout 20 Nov 13 2018 etc
drwxr-xr-x 2 501 dialout 106 Nov 13 2018 include
drwxr-xr-x 3 501 dialout 20 Nov 13 2018 lib
drwxr-xr-x 2 501 dialout 239 Nov 13 2018 libexec
-rw-r--r-- 1 501 dialout 106210 Nov 13 2018 LICENSE.txt
-rw-r--r-- 1 501 dialout 15917 Nov 13 2018 NOTICE.txt
-rw-r--r-- 1 501 dialout 1366 Nov 13 2018 README.txt
drwxr-xr-x 3 501 dialout 4096 Nov 13 2018 sbin
drwxr-xr-x 4 501 dialout 31 Nov 13 2018 share
### --- 注释说明
~~~ bin目录:对Hadoop进行操作的相关命令,如hadoop,hdfs等
~~~ etc目录:Hadoop的配置文件目录,入hdfs-site.xml,core-site.xml等
~~~ lib目录:Hadoop本地库(解压缩的依赖)
~~~ sbin目录: 存放的是Hadoop集群启动停止相关脚本,命令
~~~ share目录:Hadoop的一些jar,官方案例jar,文档等