1.1 sed
A stream editor.
1) The sed command includes many features for selecting lines to be modified and making changes only to the selected lines.
2) save sed output. e.g. sed ‘some-sed-commands’ input-file > outfile
3) Regular Expressions refer Regular Expressions.
Option
-e | Only print line number. |
-n | Not print line. Default print all lines (include edited and not edited) |
-p | Only print edited line. |
-f | Read from file rather than from command line. Use sed script. e.g. sed -f myscript.sed input_file |
Pattern
n | n is line number, search line n. |
n,m | From line n to m, e.g. 2,5 represent from line 2 to line 5 |
/pattern/ | Search pattern |
/pattern1/pattern2/augu. | Search both 2 patterns & - remember the match pattern. E.g. sed –n ‘s/nurse/”hello” &/p’ quote.txt. # origin: … nurse…, result: … hello nurse … augu. g – Global replacement, e.g. sed ‘s/\$//g’ quote.txt |
/pattern/,n | Search pattern in line n |
n,/pattern/ | As above. |
n,m! | Not search line n and m. |
p | Print match line. |
= | Display line number |
d | Remove line |
c\ | Replace text. |
s | Replace mode. |
r | Read text from another file. E.g. sed ‘/company./r file.txt’ quote.txt |
w | Write file |
q | Exit pattern if pattern matched |
n | Concatenate. |
$ | Last line. |
e.g.
sed "s/happy/enchanted/g" chap1 >chap1.new
Replaces each occurrence of the word happy found in the file chap1 with the word enchanted. Without the g character, replaces only the first occurrence of the word happy on a line.
sed –n ‘/\$/’p quote.txt # \$ stands for $
sed –n ‘1,$’p quote.txt # print full file
sed 's/.txt/.id/' quote.txt # replace .txt by .id
grep xxx /etc/passwd | sed -e ‘s/Taylor/Tailor/g;s/:/ /g’
cat xxx | head -15 | sed ‘/eiyo/,/rpm/d’
sed 's;INSERT INTO;TERMINATE INTO;'
1.2 awk
More powerful than sed, a real text programming kit – awk.
Finds lines in files that match a pattern and performs specified actions on those lines.
nawk – support more which written by original author;
gawk – support more which created by free software foundation GNU (may not exist in some system).
Option
-f | Read from file rather than from command line. |
-Fc | c as separator between fields (default space) |
Pattern
awk ‘{command}’ | Command format. |
$n | Field n. ($0 is the entire line) |
“xxx” | The constant. |
NF | how many fields are on a line (for the blank field cases) |
$NF | always the value of the last field on the line (for the blank field cases) |
NR | Record count. |
END | When pointer at the end of records. |
Command
| Display all lines. |
Symbol
= =+ -= *= /= %= ^= | Assignment. |
?: | C conditional expression (nawk and gawk). |
|| | Logical OR. |
&& | Logical AND. |
~ !~ | Match regular expression and negation. |
< <= > >= != == | Relational operators. |
(blank) | Concatenation. |
+ - | Addition, subtraction. |
* / % | Multiplication, division, and modulus. |
+ - ! | Unary plus and minus, and logical negation. |
^ | Exponentiation. |
++ -- | Increment and decrement, either prefix or postfix. |
$ | Field reference. |
String Function (awk)
Option or argument | Function |
gsub(r,s,t) | For nawk. Globally substitutes s for each match of the regular expression r in the string t. Returns the number of substitutions. If t is not supplied, defaults to $0. |
index(s,t) | Returns position of substring t in string s or zero if not present. (offset start from 1) |
length(s) | Returns length of string s or length of $0 if no string is supplied. |
match(s,r) | For nawk. Returns either the position in s where the regular expression r begins, or 0 if no occurrences are found. Sets the values of RSTART and RLENGTH. |
split(s,a,sep) | Parses string s into elements of array a using field separator sep; returns number of elements. If sep is not supplied, FS is used. Array splitting works the same way as field splitting. (array start from [1]) |
sprintf("fmt",expr) | Uses printf format specification for expr. e.g. sprintf(“%c”,i) |
sub(r,s,t) | For nawk. Substitutes s for first match of the regular expression r in the string t. Returns 1 if successful; 0 otherwise. If t is not supplied, defaults to $0. |
substr(s,p) substr(s,p,n) | Returns substring of string s at beginning position p up to a maximum length of n. If n is not supplied, the rest of the string from p is used. |
tolower(s) | For gawk. Translates all uppercase characters in string s to lowercase and returns the new string. |
toupper(s) | For gawk. Translates all lowercase characters in string s to uppercase and returns the new string. |