linux filter

cat – copy file

format:  cat textFile 或 cat < textFile

option: 
-n            显示行数(从1开始)
-s 		      合并相邻空行			  

---

wc – word counter

format:  wc [option] textFile

option: 
-c            num of chars
-l			  num of lines
-w		      num of words

---

tr – convert text char by char

format:  tr 'sourceCode' 'destCode' < textFile 

option: 
-c            sourceCode => complement of source
-d			  delete all chars in source
-s			  合并相邻重复chars

command	transformation	description
tr ‘ABC’ ‘123’ < textFile	A -> 1, B -> 2, c -> 3	一对一转换
tr ‘ABCD’ ‘123’ < textFile	A -> 1, B -> 2, c -> 3, D->3	多余的char转换成最后一个char
tr -d ‘0-9’ < textFile	0 -> ‘’, 1 -> ‘’, 2 -> ‘’	删除所有数字
tr ‘A-Z’ ‘a-z’ < textFile	A ->a, B->b, Z->z	大写转换成小写

---

head / tail – select lines

format:  head / tail  [option] [<] textFile

option: 
-n      行数（默认10行）

command	description
head textFile	print first 10 lines
tail <textFile	print last 10 lines
head -n 100 textFile	print first 100 lines
tail -n 20 textFile	print last 20 lines

结合head 和 tail,截取段落

head -n 100 textFile | tail -n 20 ----- 81-100行

---

egrep – select lines matching pattern

– 输入所有匹配pattern的行

format:  egrep [option] "pattern" [<] textFile
pattern: 通常为 regular expression

option: 
-i         忽略大小写
-v         only display lines that do not match the pattern
-w         only match pattern if it makes a complete word

一个栗子
text.txt 文件

hello world
hello 
world
hell
world hello

输入:

egrep “hello” text.txt

输出

hello world
hello 
world hello

special: egrep -v “^$” : 去掉空白行

---

cut - print selected parts of textFile

format:  cut  [option] [<] data

option: 
-f   n[-m]              nth (to mth) column (-tab seperated)
-d   'c'               use ;c' as seperator
-c   n[-m]              nth (to mth) chars

command	description
cut -f1 data	print first column
cut -f1-3 data	print 1th - 3rd columns
cut -f1,4 data	print all columns after the third
cut -c1-5 data	print the first five chars on each line
cut -d ’ : ’ -f1-3 data	print 1th - 3rd columns, seperated by “:”

---

sort – sort lines

format:  sort  [option] [<] textFile

option: 
-r           sort in descending order (reverse sort)
-n           sort numerically rather than lexicographically
-t’c’        use character c to separate columns (default: space)
-kn          sort on column n

command	description
sort -nr -k3 data	sort numbers in 3rd column in descending order
sort -t ‘:’ -k5 data	sorted by 5th column, separated by ：

---

uniq – count or remove duplicates

The uniq command by default removes all but one copy of adjacent identical lines.

format:  uniq  [option] [<] textFile

option: 
-c       显示每行重复次数
-d       只显示有重复的行
-u       只显示没有重复的行

---