【Linux】文件拆分

在工作有时需要将较大的日志文件拆分成多个较小的文件，那么我们可以用split命令

[root@localhost opt]# split --help
Usage: split [OPTION]... [INPUT [PREFIX]]
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default
size is 1000 lines, and default PREFIX is 'x'.  With no INPUT, or when INPUT
is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
  -a, --suffix-length=N   generate suffixes of length N (default 2)
      --additional-suffix=SUFFIX  append an additional SUFFIX to file names
  -b, --bytes=SIZE        put SIZE bytes per output file
  -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file
  -d, --numeric-suffixes[=FROM]  use numeric suffixes instead of alphabetic;
                                   FROM changes the start value (default 0)
  -e, --elide-empty-files  do not generate empty output files with '-n'
      --filter=COMMAND    write to shell COMMAND; file name is $FILE
  -l, --lines=NUMBER      put NUMBER lines per output file
  -n, --number=CHUNKS     generate CHUNKS output files; see explanation below
  -u, --unbuffered        immediately copy input to output with '-n r/...'
      --verbose           print a diagnostic just before each
                            output file is opened
      --help     display this help and exit
      --version  output version information and exit

SIZE is an integer and optional unit (example: 10M is 10*1024*1024).  Units
are K, M, G, T, P, E, Z, Y (powers of 1024) or KB, MB, ... (powers of 1000).

CHUNKS may be:
N       split into N files based on size of input
K/N     output Kth of N to stdout
l/N     split into N files without splitting lines
l/K/N   output Kth of N to stdout without splitting lines
r/N     like 'l' but use round robin distribution
r/K/N   likewise but only output Kth of N to stdout

GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
For complete documentation, run: info coreutils 'split invocation'
[root@localhost opt]#

通过三个例子了解split用法

例子一

[root@localhost opt]# mkdir logs
[root@localhost opt]# ls
23.txt  fastdfs_storage  fastdfs_storage_data  fastdfs_storage_data1  fastdfs_tracker  logs  testTemp
[root@localhost opt]# cd logs
[root@localhost logs]# ls
[root@localhost logs]# wc -l /opt/fastdfs_storage/logs/storaged.log
551 /opt/fastdfs_storage/logs/storaged.log
[root@localhost logs]# split -l 100 /opt/fastdfs_storage/logs/storaged.log storagedLog
[root@localhost logs]# ls
storagedLogaa  storagedLogab  storagedLogac  storagedLogad  storagedLogae  storagedLogaf
[root@localhost logs]# wc -l storagedLoga*
  100 storagedLogaa
  100 storagedLogab
  100 storagedLogac
  100 storagedLogad
  100 storagedLogae
   51 storagedLogaf
  551 total
[root@localhost logs]#

统计指定文件的总行数

wc -l /opt/fastdfs_storage/logs/storaged.log

从例子中可以看出该文件有551行，我们以每一百行为一个文件的规则将该文件拆分成多个文件，拆分后的文件名前缀为storagedLog

split -l 100 /opt/fastdfs_storage/logs/storaged.log storagedLog

注意：拆分后的多个文件与原文件不一定在同一个目录下，拆分后的多个文件保存的路径与执行split命令时所在的路径一致，在本例中，拆分后的多个文件在/opt/logs/路径下，与原文件不在同一个目录下
在例子的最后，统计了拆分后每个文件的总行数

例子二

[root@localhost logs]# wc -c /opt/fastdfs_storage/logs/storaged.log
24726 /opt/fastdfs_storage/logs/storaged.log
[root@localhost logs]# split -b 10000 /opt/fastdfs_storage/logs/storaged.log storagedTemp
[root@localhost logs]# ls
storagedLogaa  storagedLogac  storagedLogae  storagedTempaa  storagedTempac
storagedLogab  storagedLogad  storagedLogaf  storagedTempab
[root@localhost logs]# wc -c storagedTemp*
10000 storagedTempaa
10000 storagedTempab
 4726 storagedTempac
24726 total
[root@localhost logs]#

统计指定文件的字节数

wc -c /opt/fastdfs_storage/logs/storaged.log

从例子中可以看出该文件的大小为24726 byte，我们以每10000 byte为一个文件的规则将该文件拆分成多个文件，拆分后的文件名前缀为storagedTemp

split -b 10000 /opt/fastdfs_storage/logs/storaged.log storagedTemp

在例子的最后，统计了拆分后每个文件的大小

例子三