大侠们是否删除百万千万级甚至更多的小文件吧,用rm -rf 命令非常慢,极度挑战人类的耐性。这时可用rsync工具删除,测试过Linux和HP-UX都可以用。
先确定操作系统是否装好rsync,HP-UX默认未装rsync,到http://hpux.connect.org.uk/搜索rsync包下载安装即可,记得先装依赖包。
rsync使用的是替换原理。
1、创建一个空目录
-bash-4.3# mkdir /mnt/empt
2、使用rsync同步删除
注意:目录后面加上“/” ,/mnt/many/100w/是要删除的目录
-bash-4.3# rsync --delete-before -a -H -v --progress --stats /mnt/empty /mnt/many/100w/
....
deleting 1W94.3MB/300029.63MB/Lib/__future__.py
.....
Number of files: 1 (dir: 1)
Number of created files: 0
Number of deleted files: 105,720 (reg: 100,000, dir: 5,720)
Number of regular files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 46
Total bytes received: 6,339,527
sent 46 bytes received 6,339,527 bytes 7,605.97 bytes/sec
total size is 0 speedup is 0.00
....
deleting 1W94.3MB/300029.63MB/Lib/__future__.py
.....
Number of files: 1 (dir: 1)
Number of created files: 0
Number of deleted files: 105,720 (reg: 100,000, dir: 5,720)
Number of regular files transferred: 0
Total file size: 0 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 46
Total bytes received: 6,339,527
sent 46 bytes received 6,339,527 bytes 7,605.97 bytes/sec
total size is 0 speedup is 0.00
选项说明:
- --delete-before 接收者在传输之前进行删除操作
- -a 归档模式,表示以递归方式传输文件,并保持所有文件属性
- -H 保持硬连接的文件
- -v 详细输出模式
--progress 在传输时显示传输过程
- --stats 给出某些文件的传输状态
一般不需要显示进度,使用以下命令即可:
-bash-4.3# rsync --delete-before -a -H /mnt/empty /mnt/many/100w/
删除完后只剩下/mnt/many/100w/空目录,里面的文件都被删除。
3、rsync一些跟删除有关的参数
--del an alias for --delete-during
--delete delete extraneous files from destination dirs
--delete-before receiver deletes before transfer, not during
--delete-during receiver deletes during the transfer
--delete-delay find deletions during, delete after
--delete-after receiver deletes after transfer, not during
--delete-excluded also delete excluded files from destination dirs
--delete-missing-args delete missing source args from destination
--ignore-errors delete even if there are I/O errors
--max-delete=NUM don't delete more than NUM files