hadoop常用指令

最新推荐文章于 2021-11-30 11:33:08 发布

tuiyidelongdidi

最新推荐文章于 2021-11-30 11:33:08 发布

阅读量92

点赞数

分类专栏： hadoop 文章标签：大数据

本文链接：https://blog.csdn.net/tuiyidelongdidi/article/details/84551972

版权

hadoop 专栏收录该内容

19 篇文章 0 订阅

订阅专栏

1、列出正在运行的job

nange@ubuntu:~$ hadoop job -list
0 jobs currently running
JobId	State	StartTime	UserName	Priority	SchedulingInfo

2、kill掉job（1234指jobid）

nange@ubuntu:~$ hadoop job -kill 1234

3、查看hdfs块状态

nange@ubuntu:~$ hadoop fsck /
FSCK started by nange from /127.0.0.1 for path / at Mon Apr 21 22:26:29 CST 2014
................................
/hdfs/test/avg/out/part-r-00000:  Under replicated blk_958972218415296316_1141. Target Replicas is 3 but found 1 replica(s).
.
/hdfs/test/hdfsoper/2.f:  Under replicated blk_1024136243155007162_1126. Target Replicas is 3 but found 1 replica(s).

4、查看hdfs状态，并删除损坏块

nange@ubuntu:~$ hadoop fsck / -delete

5、检查dfs状态，以及datanode信息

nange@ubuntu:~$ hadoop dfsadmin -report
Configured Capacity: 155277496320 (144.61 GB)
Present Capacity: 138432897024 (128.93 GB)
DFS Remaining: 138429386752 (128.92 GB)
DFS Used: 3510272 (3.35 MB)
DFS Used%: 0%
Under replicated blocks: 14
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Datanodes available: 1 (1 total, 0 dead)

Name: 127.0.0.1:50010
Decommission Status : Normal
Configured Capacity: 155277496320 (144.61 GB)
DFS Used: 3510272 (3.35 MB)
Non DFS Used: 16844599296 (15.69 GB)
DFS Remaining: 138429386752(128.92 GB)
DFS Used%: 0%
DFS Remaining%: 89.15%
Last contact: Mon Apr 21 22:33:15 CST 2014

6、进入离开安全模式

nange@ubuntu:~$ hadoop dfsadmin -safemode enter
Safe mode is ON
nange@ubuntu:~$ hadoop dfsadmin -safemode leave
Safe mode is OFF

7、hadoop 并行拷贝（将a节点数据拷贝到b节点）

hadoop distcp hdfs://a:9000/a hdfs://b:9000/b

8、平衡集群文件

nange@ubuntu:~$ start-balancer.sh 
starting balancer, logging to /home/nange/programs/hadoopWS/hadoop-1.2.1/libexec/../logs/hadoop-nange-balancer-ubuntu.out