Hadoop集群的启动与停止

一.hadoop集群的启动与停止
(一)启动hadoop集群
1.切换到Hadoop的sbin文件目录中
cd /opt/server/hadoop/sbin/
2.格式化HDFS文件系统
hadoop namenode -format
3.格式化完成

4.启动hadoop
./start-dfs.sh或者start-dfs.sh

5.启动yarn
./strat-yarn.sh

6.使用jps查看进程,直接在下面发送全部命令,三台虚拟机分别是5  4  3台

7.进入hadoop的web网站进行访问,用自己的虚拟机IP地址
例如:192.168.191.131:9870    # ip+端口

#关闭
systemctl stop firewalld.service
#查看
systemctl status firewalld.service
#防火墙失效设置
systemctl disable firewalld.service


(二)停止hadoop集群
1.停止hadoop
./stop-dfs.sh

2.停止yarn
./stop-yarn.sh


二.编辑hadoop启动与停止的脚本
(一)编辑脚本文件
1.切换到脚本文件目录
cd /opt/script/


2.编辑hadoop.sh集群控制文件
vim hadoop.sh


3.脚本文件
#!/bin/bash
case $1 in
"start"  ){
/opt/server/hadoop/sbin/start-dfs.sh
/opt/server/hadoop/sbin/start-yarn.sh
};;
"stop"){
/opt/server/hadoop/sbin/stop-dfs.sh
/opt/server/hadoop/sbin/stop-yarn.sh
};;
esac
4.查看脚本文件是否完成创建
ll


5.授权
chmod 777 hadoop.sh
r=4    读的权限    w=2   写的权限       x=1    执行的权限

(二)启动和停止hadoop集群
1.启动hadoop集群
./hadoop.sh start
2.查看进程
jps
3.hadoop02主机查看进程
jps
4.停止hadoop集群
./hadoop.sh stop
5.查看进程
jps
6.hadoop02主机查看进程

  • 1
    点赞
  • 5
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
目录 1 ...........................................................................................................................4 概述 1.1 ..................................................................................................4 什么是Hadoop? 1.2 .......................................................................................4 为什么要选择Hadoop? 1.2.1 ........................................................................................................4 系统特点 1.2.2 ........................................................................................................5 使用场景 2 ...........................................................................................................................5 术语 3 ....................................................................................................6 Hadoop的单机部署 3.1 .....................................................................................................................6 目的 3.2 ..............................................................................................................6 先决条件 3.2.1 ........................................................................................................6 支持平台 3.2.2 ........................................................................................................6 所需软件 3.2.3 ........................................................................................................6 安装软件 3.3 .....................................................................................................................7 下载 3.4 ................................................................................7 运行Hadoop集群的准备工作 3.5 ............................................................................................7 单机模式的操作方法 3.6 .....................................................................................8 伪分布式模式的操作方法 3.6.1 ................................................................................................................8 配置 3.6.2 ................................................................................................9 免密码ssh设置 3.6.3 ................................................................................................................9 执行 4 .......................................................................................11 Hadoop集群搭建过程手记 4.1 .................................................................................................12 免密码SSH设置 4.2 ................................................................................................12 Hadoop软件安装 4.3 ..................................................................................................13 Master(85)配置 4.4 .........................................................................................14 Slave(60,245上)配置 4.5 ..................................................................................15 初始化和启动hadoop集群 4.5.1 ............................................................................................15 初始化文件系统 4.5.2 .................................................................................................15 启动Hadoop 4.5.3 .................................................................................................17 停止Hadoop 4.6 ...................................................................................................................17 测试 4.7 .................................................................................................19 管理界面与命令 4.7.1 ........................................................................................19 hdfs运行状态界面 4.7.2 .........................................................................20 Map-reduce的运行状态界面 4.7.3 ........................................................................................20 直接的命令行查看 4.7.1 ............................................................................................21 运行的进程查看 5 ..................................................................................................................22 架构分析 5.1 .................................................................................................................22 HDFS 5.1.1 ..................................................................................23 HDFS的三个重要角色 5.1.2 .............................................................................................24 HDFS设计特点 5.2 .......................................................................................................25 MapReduce www.linuxidc.com Linux公社(LinuxIDC.com) 是包括Ubuntu,Fedora,SUSE技术,最新IT资讯等Linux专业类网站。 5.2.1 ......................................................................................................25 算法介绍 5.2.2 ........................................................................27 Hadoop框架下的mapreduce 5.3 .....................................................................................................28 综合架构分析 6 .........................................................................................................37 Hadoop的应用 7 ..................................................................................................................38 系统维护 7.1 ............................................................................................38 Hadoop的系统监控 7.2 Hadoop中的命令(Command)总结.....................................错误!未定义书签。 7.3 ..............................................................38 NameNode与JobTracker单点故障说明 7.4 ............................................................................................................39 经验总结 7.5 .......................................39 如何在一个hadoop集群新增或删除一些机器而不重启 7.5.1 ......................................................................................................39 新增节点 7.5.2 ......................................................................................................40 删除节点 7.6 ..............................................................................................42 其它日常问题说明 7.6.1 ..............42 datanode启动失败,各slave节点的namespaceIDs与masters不同 7.6.2 .............................................................43 taskTracker和jobTracker 启动失败 7.6.3 ...43 Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out 7.6.4 ...............................................................................44 Too many fetch-failures 7.6.5 ..................................44 能够启动datanode,但无法访问,也无法结束的错误 7.6.6 ................................................44 java.io.IOException: Could not obtain block: 7.6.7 ..........................................44 java.lang.OutOfMemoryError: Java heap space 7.6.8 ........................................................45 解决hadoop OutOfMemoryError问题: 7.6.9 .......................................................................45 Hadoop java.io.IOException: 7.7 .......................................................................................45 防火墙的端口开放要求 7.7.1 ....................................................................45 与HDFS有关的地址及端口属性 7.7.2 .........................................................46 与MapReduce 有关的地址及端口属性 8 .........................................................................................................................47 附录 8.1 .......................................................................................................47 hadoop历史 8.2 ...................................................................................................49 Hadoop大记事 8.3 .................................................................................49 Hadoop的几个主要子项目 8.4 ..............................................................................................50 官方集群搭建参考 8.4.1 ......................................................................................................50 配置文件 8.4.2 ...............................................................................................50

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值