默认情况下，hdfs dfs -ls命令会提供未排序的文件列表。对于HDFS目录中文件的时间戳排序，目前没有内置命令，通常使用管道和sort命令实现。在Hadoop 2.7.x及更高版本中，可以使用-hdf dfs -ls -t或-hdfs dfs -ls -r -t选项按修改时间或访问时间进行排序。
Is there a hdfs command to list files in HDFS directory as per timestamp, ascending or descending? By default, hdfs dfs -ls command gives unsorted list of files.
When I searched for answers what I got was a workaround i.e. hdfs dfs -ls /tmp | sort -k6,7. But is there any better way, inbuilt in hdfs dfs commandline?
解决方案
No, there is no other option to sort the files based on datetime.
If you are using hadoop version < 2.7, you will have to use sort -k6,7 as you are doing:
hdfs dfs -ls /tmp | sort -k6,7
And for hadoop 2.7.x ls command , there are following options available :
Usage: hadoop fs -ls [-d] [-h] [-R] [-t] [-S] [-r] [-u]
Options:
-d: Directories are listed as plain files.
-h: Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864).
-R: Recursively list subdirectories encountered.
-t: Sort output by modification time (most recent first).
-S: Sort output by file size.
-r: Reverse the sort order.
-u: Use access time rather than modification time for display and sorting.
So you can easily sort the files:
hdfs dfs -ls -t -R (-r) /tmp
扫一扫
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
