【HDFS篇】HDFS的常用的命令操作汇总

一、HDFS
基本的检查操作如下：

部分举例如下：
1、查看空间使用情况：hdfs dfs -df -h
Filesystem Size Used Available Use%
hdfs://mycluster 288.8 G 34.4 M 269.2 G 0%

2、查看文件列表：hdfs dfs -ls /
Found 4 items
drwxr-xr-x - root supergroup 0 2018-01-28 21:09 /000
drwx—— - root supergroup 0 2018-01-26 01:02 /tmp
drwxr-xr-x - root supergroup 0 2018-01-28 21:33 /user
drwxr-xr-x - root supergroup 0 2018-02-27 18:19 /wc

3、查看目录下空间占用情况：hdfs dfs -du -h /
1.4 M /000
10.8 M /tmp
1.4 M /user
3.0 M /wc

4、hdfs块检查：hdfs fsck /
Connecting to namenode via http://node02:50070
FSCK started by root (auth:SIMPLE) from /192.168.159.36 for path / at Tue Feb 27 22:36:42 CST 2018
……………………………………………….Status: HEALTHY
Total size: 17471820 B
Total dirs: 19
Total files: 55
Total symlinks: 0
Total blocks (validated): 52 (avg. block size 335996 B)
Minimally replicated blocks: 52 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 2
Average block replication: 2.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 3
Number of racks: 1
FSCK ended at Tue Feb 27 22:36:42 CST 2018 in 24 milliseconds
The filesystem under path ‘/’ is HEALTHY

处理HDFS上的文件：
5、创建和推送文件：
hdfs dfs -mkdir -p /tq/input
hdfs dfs -put tq.txt /tq/input

6、删除hdfs上的文件：
hdfs dfs -rm 文件对应的路径

7、运行JAR程序包shell：

hadoop jar hadoop-mapreduce-examples-2.6.5.jar wordcount  /user/root/ccc.txt  /wc/out
（XXX.jar是程序目录，wordcount是程序入口，user/root/ccc.txt是文件输入源，wc/out是文件输出源）

7、查看HDFS状态：
hadoop dfsadmin -report
查看的内容为，每个节点的信息，还有包含的datanode，以及每个datanode的情况等。

8、离开安全模式：
hadoop dfsadmin -safemode leave

9、进入安全模式：
hadoop dfsadmin -safemode enter