Shell脚本编程---sed和gawk介绍（九）

一、文本处理

（1）sed编辑器

       sed编辑器称为流编辑器，与普通的交互式文本编辑器相对应。

sed命令的格式：

sed options script file

1》在命令行中定义编辑器命令

[root@ceph01 home]# echo "This is a test" | sed 's/test/big test/'
This is a big test

[root@ceph01 sed-gawk]# cat data 
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog

[root@ceph01 sed-gawk]# sed 's/dog/cat/' data
The quick brown fox jumps over the lazy cat
The quick brown fox jumps over the lazy cat
The quick brown fox jumps over the lazy cat
The quick brown fox jumps over the lazy cat

2》在命令行中使用多个编辑器命令

[root@ceph01 sed-gawk]# sed -e 's/brown/green/; s/dog/cat/' data
The quick green fox jumps over the lazy cat
The quick green fox jumps over the lazy cat
The quick green fox jumps over the lazy cat
The quick green fox jumps over the lazy cat

[root@ceph01 sed-gawk]# sed -e '
> s/brown/green/
> s/fox/elephant/
> s/dog/cat/' data
The quick green elephant jumps over the lazy cat
The quick green elephant jumps over the lazy cat
The quick green elephant jumps over the lazy cat
The quick green elephant jumps over the lazy cat

3》从文件读取编辑器命令

[root@ceph01 sed-gawk]# cat script1 
s/brown/green/
s/fox/elephant/
s/dog/cat/
[root@ceph01 sed-gawk]# sed -f script1 data
The quick green elephant jumps over the lazy cat
The quick green elephant jumps over the lazy cat
The quick green elephant jumps over the lazy cat
The quick green elephant jumps over the lazy cat

（2）gawk程序

gawk程序是Unix中原awk程序的GNU版本。在编辑语言内部，可以：

         定义要保存数据的变量

         使用算术和字符串操作符对数据进行计算

          使用结构化编程概念，例如if-then语句和循环，将逻辑添加到数据处理过程

          通过从数据文件内抽取数据元素以及按照其他顺序或格式对它们重定位，生成带格式的报告。

1》gawk命令格式

gawk options program file

2》自命令行读取程序脚本

[root@ceph01 sed-gawk]# gawk '{print "Hello John!"}'
This is test             
Hello John!
hello 
Hello John!

3》使用数据字段变量

gawk的主要功能之一是处理文本文件中数据的能力。

$0 表示整行文本；

$1表示文本行中的第一个数据字段；

$2表示文本行中的第一个数据字段；

$n表示文本行中的第n个数据字段；

[root@ceph01 sed-gawk]# cat data 
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
[root@ceph01 sed-gawk]# gawk '{print $1}' data
The
The
The
The
The
The
The

4》在程序脚本中使用多个命令

[root@ceph01 sed-gawk]# echo "My name is Rich" | gawk '{$4="Dave"; print $0}'
My name is Dave
[root@ceph01 sed-gawk]# gawk '{
> $4="testing"
> print $0 }'

   testing

   testing

5》从文件读取程序

[root@ceph01 sed-gawk]# cat script2 
{ print $5 "'s userid is " $1 }

[root@ceph01 sed-gawk]# gawk -F: -f script2 data
's userid is The quick brown fox jumps over the lazy dog
's userid is The quick brown fox jumps over the lazy dog
's userid is The quick brown fox jumps over the lazy dog
's userid is The quick brown fox jumps over the lazy dog
's userid is The quick brown fox jumps over the lazy dog
's userid is The quick brown fox jumps over the lazy dog
's userid is The quick brown fox jumps over the lazy dog

6》在处理数据之前运行脚本

[root@ceph01 sed-gawk]# gawk 'BEGIN {print "Hello World!"} {print $0}'
Hello World!
This is test       
This is test
this is another test
this is another test

7》在处理数据之后运行脚本

[root@ceph01 sed-gawk]# gawk 'BEGIN {print "Hello World!"} {print $0} END {print "byebye"}'
Hello World!
this is a test
this is a test
byebye

二、sed编辑器基础知识

（1）更多替换选项

替换标记：

s/pattern/replacement/flags

可用的替换标记有4种:

       数字 ：表示新文本替换的模式

       g      ：表示用新文本替换现有文本的全部实例

       p      ：表示打印原始行的内容

       w file：将替换的结果写入文件

[root@ceph01 sed-gawk]# sed 's/fox/cat/g' data
The quick brown cat jumps over the lazy dog
The quick brown cat jumps over the lazy dog
The quick brown cat jumps over the lazy dog

替换字符

$:sed 's!/bin/bash!/bin/csh!' /etc/passwd

（2）使用地址

在sed编辑器中，有两种寻址形式：

        行的数值范围

        筛选行的文本模式

格式：

[address] command

--------------------------------------------------------------------------
address {
    command1
    command2
    command3
}

1》数字式行寻址

[root@ceph01 sed-gawk]# sed '2s/dog/cat/' data
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy cat
The quick brown fox jumps over the lazy dog

2》使用文本模式筛选器

命令格式：

/pattern/command

3》组合命令

[root@ceph01 sed-gawk]# sed '2{
> s/fox/elephant/
> s/dog/cat/
> }' data
The quick brown fox jumps over the lazy dog
The quick brown elephant jumps over the lazy cat
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog

（3）删除行

删除指定的行

$:sed '3d' data

删除特定的文本行范围

$:sed '2,3d' data

（4）插入和附加文本

     插入命令（i）在指定行之前添加新的一行；

     附加命令（a）在指定行之后添加新的一行

命令格式：

sed '[address]command new line'

[root@ceph01 sed-gawk]# echo "testing" | sed 'i\
> this is a test'
this is a test
testing
[root@ceph01 sed-gawk]# echo "testing" | sed 'a\
this is a test'
testing
this is a test

（5）更改行

[root@ceph01 sed-gawk]# sed '3c\
> this is a changed line of text.' data
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
this is a changed line of text.
The quick brown fox jumps over the lazy dog

（6）变换命令

变换命令（y）是唯一对单个字符进行操作的sed编辑器命令。命令格式：

[address]y/inchars/outchars/

[root@ceph01 sed-gawk]# sed 'y/123/789/' data
This is line number 7
This is line number 8
This is line number 9
This is line number 4
This is line number 5

（7）打印命令温习

         打印文本行的小写p命令

         打印行号的等号（=）命令

         列出行的1（小写L）命令

1》打印行

[root@ceph01 sed-gawk]# echo "this is test" | sed 'p'
this is test
this is test

2》打印行号

[root@ceph01 sed-gawk]# sed '=' data 
1
The quick brown fox jumps over the lazy dog
2
The quick brown fox jumps over the lazy dog
3
The quick brown fox jumps over the lazy dog
4
The quick brown fox jumps over the lazy dog

3》列出行

[root@ceph01 sed-gawk]# sed -n 'l' data
The quick brown fox jumps over the lazy dog$
The quick brown fox jumps over the lazy dog$
The quick brown fox jumps over the lazy dog$

（8）将文件用于sed

1》写文件

w命令用于将文本行写入文件：

[address]w filename

[root@ceph01 sed-gawk]# sed '1,2w test' data
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog

[root@ceph01 sed-gawk]# cat test 
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog

2》从文件读取数据

[address]r filename

[root@ceph01 sed-gawk]# cat test 
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
[root@ceph01 sed-gawk]# sed '2r test' data
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog
The quick brown fox jumps over the lazy dog