【Windows】
1、本地拷贝推荐:FastCopy
2、网络拷贝推荐:RichCopy
【Linux】
1、rsync 命令
#删除海量文件目录 mkdir empty_dir rsync -a --delete empty_dir/ yourdirectory/ #本地拷贝 rsync -a /some/path/to/src/ /other/path/to/dest/ #网络拷贝 rsync -a -e ssh /path/to/dir/ remote_server:/path/to/remotedir #网速IO慢于磁盘IO rsync -azP -e ssh /path/to/dir/ remote_server:/path/to/remotedir
2、tar 命令结合管道
#本地拷贝(-C 前面的短横表示解包文件来自标准输入) tar -c /path/to/dir | tar -xvf - -C /absolute/path/to/remotedir #网络拷贝 tar -c /path/to/dir | ssh remote_server 'tar -xvf - -C /absolute/path/to/remotedir'
3、rsync 统计值
$ rsync -a --stats --dry-run /lib ~/fake Number of files: 19,375 (reg: 16,015, dir: 3,052, link: 308) Number of created files: 19,375 (reg: 16,015, dir: 3,052, link: 308) Number of deleted files: 0 Number of regular files transferred: 16,015 Total file size: 830,561,043 bytes #含普通文件、链接文件的大小,目录大小计数为0 Total transferred file size: 830,555,045 bytes #普通文件大小 Literal data: 0 bytes Matched data: 0 bytes File list size: 0 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds Total bytes sent: 471,220 Total bytes received: 61,277 sent 471,220 bytes received 61,277 bytes 1,064,994.00 bytes/sec total size is 830,561,043 speedup is 1,559.75 (DRY RUN) # 普通文件大小 # find 的 print0 参数与 xargs -0 参数是为了避免文件名中的特殊字符 $ sudo find /lib -type f -print0 | xargs -0 ls -lad | awk -F ' ' 'BEGIN{sum=0} {sum+=$5} END{printf "%d bytes\n", sum}' 830555045 bytes # 目录文件大小 # 注意目录文件大小与目录一级文件数量有关,可能大于4096(磁盘块大小) # 所以 4096 * 目录个数 <= 目录文件大小之和 $ sudo find /lib -type d -print0 | xargs -0 ls -lad | awk -F ' ' 'BEGIN{sum=0} {sum+=$5} END{printf "%d bytes\n", sum}' 12656640 bytes # 链接文件大小 $ sudo find /lib -type l -print0 | xargs -0 ls -lad | awk -F ' ' 'BEGIN{sum=0} {sum+=$5} END{printf "%d bytes\n", sum}' 5998 bytes
【相关阅读】
*** walker ***
转载于:https://blog.51cto.com/walkerqt/1958382