linux文件比较,合并,查找重复行

关键命令

sort, uniq, cat

合并多个文件

cat file1 file2 > newFile

查看文件中重复行

sort file1|uniq -d

查看文件中唯一行

sort file|uniq -u

更详细的命令，请查看sort, uniq, cat命令的解释

uniq --help

$uniq --help
Usage: uniq [OPTION]... [INPUT [OUTPUT]]
Discard all but one of successive identical lines from INPUT (or
standard input), writing to OUTPUT (or standard output).

Mandatory arguments to long options are mandatory for short options too.
  -c, --count           prefix lines by the number of occurrences
  -d, --repeated        only print duplicate lines
  -D, --all-repeated[=delimit-method] print all duplicate lines
                        delimit-method={none(default),prepend,separate}
                        Delimiting is done with blank lines.
  -f, --skip-fields=N   avoid comparing the first N fields
  -i, --ignore-case     ignore differences in case when comparing
  -s, --skip-chars=N    avoid comparing the first N characters
  -u, --unique          only print unique lines
  -w, --check-chars=N   compare no more than N characters in lines
      --help     display this help and exit
      --version  output version information and exit

转载于:https://www.cnblogs.com/forward/archive/2010/04/24/1719363.html