Hadoop安装部署
1、 软件包上传并解压
a) tar -zxvf hadoop-2.6.0-cdh5.14.0-with-centos6.9.tar.gz -C …/servers/
2、 第二步查看hadoop支持的压缩方式以及本地库
a) ./hadoop checknative
b) 安装openssl
yum -y install openssl-devel
./hadoop checknative
3、 修改配置文件
a) core-site.xml
b) hdfs-site.xml
c) Hadoop-env.sh (不用改)
d) mapred-site.xml
i. cp mapred-site.xml.template mapred-site.xml
ii. 修改
e) yarn-site.xml
f) slaves
4、 创建文件夹
mkdir -p /export/servers/hadoop-2.6.0-cdh5.14.0/hadoopDatas/tempDatas
mkdir -p /export/servers/hadoop-2.6.0-cdh5.14.0/hadoopDatas/namenodeDatas
mkdir -p /export/servers/hadoop-2.6.0-cdh5.14.0/hadoopDatas/datanodeDatas
mkdir -p /export/servers/hadoop-2.6.0-cdh5.14.0/hadoopDatas/dfs/nn/edits
mkdir -p /export/servers/hadoop-2.6.0-cdh5.14.0/hadoopDatas/dfs/snn/name
mkdir -p /export/servers/hadoop-2.6.0-cdh5.14.0/hadoopDatas/dfs/nn/snn/edits
#### 5、安装包的分发
scp -r hadoop-2.6.0-cdh5.14.0/ node02:$PWD
scp -r hadoop-2.6.0-cdh5.14.0/ node03:$PWD
6、配置hadoop环境变量
a) 创建文件/etc/profile.d/hadoop.sh 并编辑
export HADOOP_HOME=/export/servers/hadoop-2.6.0-cdh5.14.0
export PATH=
P
A
T
H
:
PATH:
PATH:HADOOP_HOME/bin
b) source /etc/profile
7、 启动集群
a) 格式化集群
在第一个节点执行 hadoop namenode -format
b) 启动
i. 每个组件独立启动
-
启动namenode node01节点 ./hadoop-daemon.sh start namenode
-
启动namenode node01、02、03节点 ./hadoop-daemon.sh start datanode
-
启动resourcemanager node01节点 ./yarn-daemon.sh start resourcemanager
-
启动nodemanager node01、02、03节点 ./yarn-daemon.sh start nodemanager
ii. 单独启动
- 单独启动HDFS : ./start-dfs.sh 关闭 ./stop-dfs.sh
- 单独启动Yarn : ./start-yarn.sh 关闭 ./stop-yarn.sh
iii. 一键启动所有
启动: ./start-all.sh 关闭:./stop-all.sh
8、浏览器查看启动页面
http://主节点ip:50070
http://主节点ip:8088