相关知识
调用文件系统(FS)Shell命令应使用 hadoop fs <args>的形式。 所有的的FS shell命令使用URI路径作为参数。URI格式是scheme://authority/path。对HDFS文件系统,scheme是hdfs,对本地文件系统,scheme是file。其中scheme和authority参数都是可选的,如果未加指定,就会使用配置中指定的默认scheme。一个HDFS文件或目录比如/parent/child可以表示成hdfs://namenode:namenodeport/parent/child,或者更简单的/parent/child(假设你配置文件中的默认值是namenode:namenodeport)。大多数FS Shell命令的行为和对应的Unix Shell命令类似,出错信息会输出到stderr,其他信息输出到stdout。
常用的hdfs操作
- -ls——显示目录信息
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -ls / Found 5 items drwxr-xr-x - adming supergroup 0 2022-09-04 16:52 /sanguo drwxrwx--- - adming supergroup 0 2022-09-02 15:49 /tmp drwxr-xr-x - adming supergroup 0 2022-08-21 17:17 /wcinput drwxr-xr-x - adming supergroup 0 2022-09-02 15:50 /wcoutput1 drwxr-xr-x - adming supergroup 0 2022-08-21 17:47 /wcoutput2 [adming@hadoop101 hadoop-3.1.3]$
- -cat——显示文件内容
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -cat /sanguo/wuguo.txt 2022-09-04 17:36:06,120 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false wuguo
- -chgrp、chmod、-chown:Linux文件系统中的用法一样,修改文件所属权限
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -chmod 666 /sanguo/wuguo.txt [adming@hadoop101 hadoop-3.1.3]$ hadoop fs -chown adming:adming /sanguo/wuguo.txt
- -mkdir——创建路径
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -mkdir /jinguo [adming@hadoop101 hadoop-3.1.3]$ hadoop fs -ls / Found 6 items drwxr-xr-x - adming supergroup 0 2022-09-04 17:38 /jinguo drwxr-xr-x - adming supergroup 0 2022-09-04 16:52 /sanguo drwxrwx--- - adming supergroup 0 2022-09-02 15:49 /tmp drwxr-xr-x - adming supergroup 0 2022-08-21 17:17 /wcinput drwxr-xr-x - adming supergroup 0 2022-09-02 15:50 /wcoutput1 drwxr-xr-x - adming supergroup 0 2022-08-21 17:47 /wcoutput2
- -cp——从HDFS的一个路径拷贝到另一个HDFS路径
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -cp /sanguo/shuguo.txt /jinguo 2022-09-04 17:42:04,294 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2022-09-04 17:42:06,179 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false [adming@hadoop101 hadoop-3.1.3]$ hadoop fs -ls /jinguo Found 1 items -rw-r--r-- 3 adming supergroup 14 2022-09-04 17:42 /jinguo/shuguo.txt
- -mv——在HDFS目录中移动文件
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -ls /jinguo Found 1 items -rw-r--r-- 3 adming supergroup 14 2022-09-04 17:42 /jinguo/shuguo.txt [adming@hadoop101 hadoop-3.1.3]$ hadoop fs -mv /sanguo/weiguo.txt /jinguo [adming@hadoop101 hadoop-3.1.3]$ hadoop fs -ls /jinguo Found 2 items -rw-r--r-- 3 adming supergroup 14 2022-09-04 17:42 /jinguo/shuguo.txt -rw-r--r-- 3 adming supergroup 7 2022-09-04 16:48 /jinguo/weiguo.txt [adming@hadoop101 hadoop-3.1.3]$ hadoop fs -ls /sanguo Found 2 items -rw-r--r-- 3 adming supergroup 14 2022-09-04 17:08 /sanguo/shuguo.txt -rw-rw-rw- 3 adming adming 6 2022-09-04 16:52 /sanguo/wuguo.txt
- -tail——显示文件末尾1kb的数据
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -tail /sanguo/shuguo.txt 2022-09-04 17:46:47,984 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false liubei liubei
- -rm——删除文件或文件夹
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -rm /sanguo/shuguo.txt Deleted /sanguo/shuguo.txt
- -rm -r:递归删除文件夹中的所有内容
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -rm -r /sanguo Deleted /sanguo
- -du——统计文件夹的大小信息
[adming@hadoop101 hadoop-3.1.3]$ hadoop fs -du /jinguo 14 42 /jinguo/shuguo.txt 7 21 /jinguo/weiguo.txt