HDFS shell 命令

HDFS FileSystem Shell
All FS shell commands take path URIs as arguments. The URI format is scheme://authority/path. For HDFS the scheme is hdfs, and for the Local FS the scheme is file. The scheme and authority are optional. 


appendToFile
 hdfs dfs -appendToFile <localsrc> ... <dst>
 hdfs dfs -appendToFile localfile hdfs://nn.example.com/hadoop/hadoopfile
 Returns 0 on success and 1 on error.


cat:Copies source paths to stdout.
 hdfs dfs -cat URI [URI ...]
 hdfs dfs -cat hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2
 hdfs dfs -cat file:///file3 /user/hadoop/file4
 Returns 0 on success and -1 on error.


chgrp:Change group association of files,改变文件所属的组,改变文件组权 change group
 hdfs dfs -chgrp [-R] GROUP URI [URI ...]
 The -R option will make the change recursively through the directory structure.
chmod:Change the permissions of files--change mode
 hdfs dfs -chmod [-R] <MODE[,MODE]... | OCTALMODE> URI [URI ...]
chown:Change the owner of files
 hdfs dfs -chown [-R] [OWNER][:[GROUP]] URI [URI ]


copyFromLocal:Similar to put command, except that the source is restricted to a local file reference.
 hdfs dfs -copyFromLocal <localsrc> URI
 The -f option will overwrite the destination if it already exists.
copyToLocal:Similar to get command, except that the destination is restricted to a local file reference.
 hdfs dfs -copyToLocal [-ignorecrc] [-crc] URI <localdst>


count:Count the number of directories, files and bytes under the paths that match the specified file pattern
The output columns with -count are: DIR_COUNT, FILE_COUNT, CONTENT_SIZE FILE_NAME
The output columns with -count -q are: QUOTA, REMAINING_QUATA, SPACE_QUOTA, REMAINING_SPACE_QUOTA, DIR_COUNT, FILE_COUNT, CONTENT_SIZE, FILE_NAMEquota:配额；定额；限额 
The -h option shows sizes in human readable format. h-humanity
 hdfs dfs -count [-q] [-h] <paths>
 hdfs dfs -count hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2
 hdfs dfs -count -q hdfs://nn1.example.com/file1
 hdfs dfs -count -q -h hdfs://nn1.example.com/file1
 Returns 0 on success and -1 on error.
cp:Copy files from source to destination. This command allows multiple sources as well in which case the destination must be a directory
 hdfs dfs -cp [-f] [-p | -p[topax]] URI [URI ...] <dest>topax:timestamps, ownership, permission, ACL, XAttr ACL容许机舱负荷（Allowance Cabin Load）
 hdfs dfs -cp /user/hadoop/file1 /user/hadoop/file2
 hdfs dfs -cp /user/hadoop/file1 /user/hadoop/file2 /user/hadoop/dir


du:Displays sizes of files and directories contained in the given directory or the length of a file in case its just a file.
 hdfs dfs -du [-s] [-h] URI [URI ...]
 The -s option will result in an aggregate summary of file lengths being displayed, rather than the individual files.
 The -h option will format file sizes in a "human-readable" fashion (e.g 64.0m instead of 67108864)
dus:This command is deprecated. Instead use hdfs dfs -du -s.


expunge:Empty the Trash(垃圾).


get：Copy files to the local file system. Files that fail the CRC check may be copied with the -ignorecrc option. Files and CRCs may be copied using the -crc option.crc循环冗余码校验
 hdfs dfs -get [-ignorecrc] [-crc] <src> <localdst>
 hdfs dfs -get /user/hadoop/file localfile
 hdfs dfs -get hdfs://nn.example.com/user/hadoop/file localfile
getfacl:Displays the Access Control Lists (ACLs:访问控制列表) of files and directories. If a directory has a default ACL, then getfacl also displays the default ACL. 
 hdfs dfs -getfacl [-R] <path>
 hdfs dfs -getfacl /file
 hdfs dfs -getfacl -R /dir


getfattr:Displays the extended attribute names and values (if any) for a file or directory
 hdfs dfs -getfattr [-R] -n name | -d [-e en] <path>
 hdfs dfs -getfattr -d /file
 hdfs dfs -getfattr -R -n user.myAttr /dir
getmerge:Takes a source directory and a destination file as input and concatenates files in src into the destination local file. 
  hdfs dfs -getmerge <src> <localdst> [addnl]
ls:
 hdfs dfs -ls [-R] <args>
 hdfs dfs -ls /user/hadoop/file1
lsr:This command is deprecated. Instead use hdfs dfs -ls -R


mkdir:Takes path uri's as argument and creates directories.
moveFromLocal:Similar to put command, except that the source localsrc is deleted after it's copied.
 hdfs dfs -moveFromLocal <localsrc> <dst>
moveToLocal:Displays a "Not implemented yet" message.
 hdfs dfs -moveToLocal [-crc] <src> <dst>
mv:Moves files from source to destination.
 hdfs dfs -mv /user/hadoop/file1 /user/hadoop/file2
put:Copy single src, or multiple srcs from local file system to the destination file system. Also reads input from stdin and writes to destination file system.


rm:Delete files specified as args.
 hdfs dfs -rm [-f] [-r|-R] [-skipTrash] URI [URI ...]
 hdfs dfs -rm hdfs://nn.example.com/file /user/hadoop/emptydir