去重复和排序【sort】【uniq】

Bash综合应用uniq sort

一、Uniq

[nick@d01 bash]$ cat uniq.txt boy took bat home boy took bat home girl took bat home dog brought hat home dog brought hat home dog brought hat home

 

（1）      uniq file

[nick@d01 bash]$ uniq uniq.txt boy took bat home girl took bat home dog brought hat home

 

Cut the repetitive lines. Andthen print out.

 

 

(2) uniq –c file

[nick@d01 bash]$ uniq -c uniq.txt       2 boy took bat home       1 girl took bat home       3 dog brought hat home

 

Cut the repetitive lines. And then printout with the repetitive line number.

 

(3) uniq –d file

[nick@d01 bash]$ uniq -d uniq.txt boy took bat home dog brought hat home

 

Only print out the repetitive lines.

 

(4) uniq –u file

[nick@d01 bash]$ uniq -u uniq.txt girl took bat home

 

Only print out the unrepetitive lines.

 

(5) uniq –f num file

[nick@d01 bash]$ uniq -f 1 uniq.txt boy took bat home dog brought hat home

 

Donot put the previous num fields intoconsideration. And then “uniq –f num file” is like “uniq file”: Cut the repetitive lines. And then print out.

 

(6) uniq –s num file

[nick@d01 bash]$ cat uniq2.txt 123 224 [nick@d01 bash]$ uniq uniq2.txt 123 223 [nick@d01 bash]$ uniq -s 1 uniq2.txt 123

Donot consider the first character of everyline. The “123” and “223” are regarded as the same lines. So: Cut the repetitive lines. And then print out.

 

 

(7) uniq –f num –s num file

[nick@d01 bash]$ cat uniq.txt boy took bat home boy took bat home girl took bat home dog brought hat home dog brought hat home dog brought hat home [nick@d01 bash]$ uniq -f 2 -s 2 uniq.txt boy took bat home

 

Donot consider the previous two fields. Andthen donot consider the previous two characters based on before result. Alllines become “at home” now. So: Cut the repetitive lines. And then print out.

 

二、Sort

（1）sort file

[nick@d01 bash]$ cat sort.txt one two three four five   [nick@d01 bash]$ sort sort.txt five four one three two   [nick@d01 bash]$ sort sort.txt -r two three one four five   [nick@d01 bash]$ sort sort.txt -r | head -n 2 two three

 

（2）      sort –n file

[nick@d01 bash]$ cat sort2.txt 1 2 100 45 3 10 145 75 [nick@d01 bash]$ sort sort2.txt 1 10 100 145 2 3 45 75 [nick@d01 bash]$ sort -n sort2.txt 1 2 3 10 45 75 100 145

 

（3）      sort –M file

[nick@d01 bash]$ cat sort3.txt Apr Aug Dec Feb Jan Jul Jun Mar May Nov Oct Sep [nick@d01 bash]$ sort sort3.txt Apr Aug Dec Feb Jan Jul Jun Mar May Nov Oct Sep [nick@d01 bash]$ sort -M sort3.txt Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

 

（4）      sort –t ‘….’ –k num file

[nick@d01 bash]$ sort -t ':' -k 3 sort4.txt root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin   [nick@d01 bash]$ sort sort4.txt bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin root:x:0:0:root:/root:/bin/bash [nick@d01 bash]$ cat sort4.txt daemon:x:2:2:daemon:/sbin:/sbin/nologin root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin

 

-t indicates the separator symbol.

-k indicates the number of field that allrecords are sorted by.

 

三、Countthe Number of All Types of Shell in Passwd File

[root@vmlink etc]# cat passwd root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin sync:x:5:0:sync:/sbin:/bin/sync shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown halt:x:7:0:halt:/sbin:/sbin/halt mail:x:8:12:mail:/var/spool/mail:/sbin/nologin news:x:9:13:news:/etc/news: uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin operator:x:11:0:operator:/root:/sbin/nologin games:x:12:100:games:/usr/games:/sbin/nologin gopher:x:13:30:gopher:/var/gopher:/sbin/nologin ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin nobody:x:99:99:Nobody:/:/sbin/nologin rpc:x:32:32:Portmapper RPC user:/:/sbin/nologin vcsa:x:69:69:virtual console memory owner:/dev:/sbin/nologin nscd:x:28:28:NSCD Daemon:/:/sbin/nologin sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin rpm:x:37:37::/var/lib/rpm:/bin/bash mailnull:x:47:47::/var/spool/mqueue:/sbin/nologin smmsp:x:51:51::/var/spool/mqueue:/sbin/nologin rpcuser:x:29:29:RPC Service User:/var/lib/nfs:/sbin/nologin nfsnobody:x:65534:65534:Anonymous NFS User:/var/lib/nfs:/sbin/nologin pcap:x:77:77::/var/arpwatch:/sbin/nologin   [root@vmlink etc]# awk -F: {'if($7 != "") print $7'} ./passwd | uniq -c | sort -n -k 1       1 /bin/bash       1 /bin/bash       1 /bin/sync       1 /sbin/halt       1 /sbin/shutdown       4 /sbin/nologin       5 /sbin/nologin      11 /sbin/nologin [root@vmlink etc]# awk -F: {'print $7'} ./passwd | uniq -c | sort -n -k 1       1       1 /bin/bash       1 /bin/bash       1 /bin/sync       1 /sbin/halt       1 /sbin/nologin       1 /sbin/shutdown       4 /sbin/nologin       5 /sbin/nologin      10 /sbin/nologin

 

四、joincommad

 

[nick@d01 bash]$cat employer.txt 100 Jason Smith 200 John Doe 300 Sanjay Gupta 400 Ashok Sharma [nick@d01 bash]$ cat bonus.txt 100 $5,000 200 $500 300 $3,000 400 $1,250 [nick@d01 bash]$ join employer.txt bonus.txt 100 Jason Smith $5,000 200 John Doe $500 300 Sanjay Gupta $3,000 400 Ashok Sharma  $1,250 [nick@d01 bash]$ join employer.txt bonus.txt | sort -k 2 400 Ashok Sharma  $1,250 100 Jason Smith $5,000 200 John Doe $500 300 Sanjay Gupta $3,000