uniq 命令去重,只能去掉连续的重复的项
test 测试文本
[shuohai.lhl@v035114 bin]$ cat test
711[10.235.170.22:8123] 0 10.232.20.178 1648042 1648042 0 2013-07-29 20:01:35
711[10.235.170.22:8123] 11 10.232.20.178 6445135 6129386 315749 2013-07-29 20:19:40
711[10.235.170.22:8123] 129 10.232.20.178 2220052 2220052 0 2013-07-29 19:33:04
711[10.235.170.22:8123] 111 10.232.20.178 4939938 4754688 185250 2013-07-29 20:19:40
711[10.235.170.22:8123] 123 10.232.20.178 284227 284227 0 2013-07-29 19:33:00
711[10.235.170.22:8123] 112 10.232.20.178 4393638 4262994 130644 2013-07-29 20:19:40
711[10.235.170.22:8123] 121 10.232.20.178 413891 413891 0 2013-07-29 19:33:00
711[10.235.170.22:8123] 130 10.232.20.178 2803552 2803552 0 2013-07-29 19:33:04
711[10.235.170.22:8123] 10 10.232.20.178 6558889 6235520 323369 2013-07-29 20:19:40
711[10.235.170.22:8123] 106 10.232.20.178 5080974 4886037 194937 2013-07-29 20:19:40
711[10.235.170.22:8123] 101 10.232.20.178 3655253 3655253 0 2013-07-29 20:19:20
711[10.235.170.22:8123] 124 10.232.20.178 424403 424403 0 2013-07-29 19:33:05
711[10.235.170.22:8123] 12 10.232.20.178 5507575 5271088 236487 2013-07-29 20:19:40
711[10.235.170.22:8123] 115 10.232.20.178 1936544 1936544 0 2013-07-29 20:04:40
711[10.235.170.22:8123] 100 10.232.20.178 2791119 2791119 0 2013-07-29 20:13:20
711[10.235.170.22:8123] 119 10.232.20.178 829978 829978 0 2013-07-29 19:48:05
711[10.235.170.22:8123] 109 10.232.20.178 5205710 4994482 211228 2013-07-29 20:19:40
711[10.235.170.22:8123] 114 10.232.20.178 2174793 2174793 0 2013-07-29 20:07:05
711[10.235.170.22:8123] 118 10.232.20.178 1119162 1119162 0 2013-07-29 19:54:20
711[10.235.170.22:8123] 113 10.232.20.178 2965206 2965206 0 2013-07-29 20:14:40
711[10.235.170.22:8123] 103 10.232.20.178 5642924 5396261 246663 2013-07-29 20:19:40
711[10.235.170.22:8123] 104 10.232.20.178 6269570 5970006 299564 2013-07-29 20:19:40
711[10.235.170.22:8123] 105 10.232.20.178 5939748 5665489 274259 2013-07-29 20:19:40
711[10.235.170.22:8123] 107 10.232.20.178 4796367 4623086 173281 2013-07-29 20:19:40
711[10.235.170.22:8123] 117 10.232.20.178 1662411 1662411 0 2013-07-29 20:01:40
711[10.235.170.22:8123] 125 10.232.20.178 283898 283898 0 2013-07-29 19:33:05
711[10.235.170.22:8123] 1 10.232.20.178 1669975 1669975 0 2013-07-29 20:01:45
711[10.235.170.22:8123] 126 10.232.20.178 1133277 1133277 0 2013-07-29 19:33:05
711[10.235.170.22:8123] 110 10.232.20.178 5399121 5176800 222321 2013-07-29 20:19:40
711[10.235.170.22:8123] 13 10.232.20.178 4599532 4599532 0 2013-07-29 19:33:04
711[10.235.170.22:8123] 116 10.232.20.178 1738524 1738524 0 2013-07-29 20:02:25
711[10.235.170.22:8123] 102 10.232.20.178 4360832 4230019 130813 2013-07-29 20:19:40
711[10.235.170.22:8123] 128 10.232.20.178 2534709 2534709 0 2013-07-29 19:33:04
711[10.235.170.22:8123] 120 10.232.20.178 715130 715130 0 2013-07-29 19:43:15
711[10.235.170.22:8123] 108 10.232.20.178 4900775 4720912 179863 2013-07-29 20:19:40
711[10.235.170.22:8123] 127 10.232.20.178 1386770 1386770 0 2013-07-29 19:33:04
711[10.235.170.22:8123] 122 10.232.20.178 424419 424419 0 2013-07-29 19:33:05
711[10.235.170.22:8123] 178 10.232.24.46 2962807 2962807 0 2013-07-29 19:33:05
打印第六列中不同值,先sort,然后在uniq
[shuohai.lhl@v035114 bin]$ cat test| awk '{print $6}' | sort |uniq -c
24 0
1 130644
1 130813
1 173281
1 179863
1 185250
1 194937
1 211228
1 222321
1 236487
1 246663
1 274259
1 299564
1 315749
1 323369