查找和替换之sed流编辑器详解



查找和替换—sed流编辑器详解

sed ：Stream Editor           

        行编辑器（全屏编辑器：vi）                    

sed 模式空间              

        默认不编辑原文件，仅对模式空间中的数据作处理:

        而后，处理结束后，将模式空间打印到屏幕   

$ sed 's/:.*//' /etc/passwd | sort -u

        删除第一个冒号之后的所有东西，排序列表并删除重复部分

       

 

1）.sed基本用法：

sed [-n] 'editing command' [file...]

sed [-n] -e 'editing command' [file...]

sed [-n] -f scripyt-file ... [file...]

       

        -n :静默模式，不再默认显示模式空间中的内容

        -i:直接修改原文件

        -f /PATH/TO/SED_SCRIPT；    自script-file中读取编辑命令

                $ sed -f /path/to/scripts file         

        -e SCRIPT -e SCRIPT:可以同时执行多个脚本

        -r :表示使用扩展正则表达式

                # history | sed ' s#[[:space:]]##g'

                # history | sed ' s#^[[:space:]]*##g' | cut -d' ' -f1

 

练习： 建立一个脚本，将/home/text目录结构建立一份副本在/home/tmp1下       

find /home/text -type d -print              #寻找所在目录

sed 's;/home/text/;/home/tmp1/;' #修改名称；注意，这里使用；作为定界符

sed 's/^/mkdir/'                                      #插入mkdir命令

sh -x                                                        #以shell跟踪模式执行

#这个脚本使用了产生命令的手法，使命令内容成为shell的输入，是个很强且常见的技巧

 

2）.替换细节：

①局部替换和全局替换

[gz_fieldyang@ test ~]$ echo Will reads well, Will writes well,Will sing well. >example.txt   

        #输入样本

[gz_fieldyang@ test ~]$ cat example.txt             #查看样本内容

Will reads well, Will writes well,Will sing well.

[gz_fieldyang@ test ~]$ sed 's/Will/Field/' <example.txt              #局部替换     

Field reads well, Will writes well,Will sing well.

[gz_fieldyang@ test ~]$ sed 's/Will/Field/2' <example.txt

#替换第n个，此处替换第二个

Will reads well, Field writes well,Will sing well.

[gz_fieldyang@ test ~]$ sed 's/Will/Field/2,3' <example.txt         #不存在此种替换

sed: -e expression #1, char 15: unknown option to `s'

[gz_fieldyang@ test ~]$ sed 's/Will/Field/3' <example.txt             #替换第三个

Will reads well, Will writes well,Field sing well.

[gz_fieldyang@ test ~]$ sed 's/Will/Field/g' <example.txt   

 #S命令以g结尾，表示全局替换

Field reads well, Field writes well,Field sing well.

 

②&在替代文本的应用与字符转义

如果你不是非常清楚你将找到的是串是什么，可以利用一个特定的字符”&”，它即指代匹配的模式

 sed ‘s/[a-z]*/(&)/’ < old > new

可以在替换时多次使用”&”，比如，你可以次每行开头的数字复制一次，如下：

$ echo “123 abc” | sed ‘s/[0-9]*/& &/’

123 123 abc

因为sed会以贪婪的方式匹配第一个串。对于’[0-9]*’的第一个区配是第一个字符，

因为这个正则是匹配0个或多个数字。所以如果输入是”abc 123”，输出也不会改变

一个能保证能匹配一个数字的更好的复制数字的方法是：

% echo “123 abc” | sed ‘s/[0-9][0-9]*/& &/’

123 123 abc

 

实例:

[gz_fieldyang@ test ~]$ sed 's/Will/&,Field/g' <example.txt

Will,Field reads well, Will,Field writes well,Will,Field sing well.  

[gz_fieldyang@ test ~]$ sed 's/Will/&,&,Field/g' <example.txt

Will,Will,Field reads well, Will,Will,Field writes well,Will,Will,Field sing well.

[gz_fieldyang@ test ~]$ sed 's/Will/&,Jones,Field/g' <example.txt 

Will,Jones,Field reads well, Will,Jones,Field writes well,Will,Jones,Field sing well.

        # &在替代文本中表示的意思是“从此点开始替代成匹配于正则表达式的整个文本”可以多次使用

[gz_fieldyang@ test ~]$ sed   's/Will/\&&/' <example.txt     

&Will reads well, Jones writes well,Kin sing well.

[gz_fieldyang@ test ~]$ sed 's/Will/\&\$/' <example.txt

&$ reads well, Jones writes well,Kin sing well      

[gz_fieldyang@ test ~]$ sed 's/Will/\&\/etc\/example\/zhuangyizufu/' <example.txt

&/etc/example/zhuangyizufu reads well, Jones writes well,Kin sing well.

                #如果要在文本中使用&,$,/等字符的字面意义，则使用反斜杠(\)转义 

       

③sed一次执行多个命令

ⅰ.sed -e ：多个命令执行

[gz_fieldyang@ test ~]$ echo Will reads well, Will writes well,Will sing well. >example.txt   

[gz_fieldyang@ test ~]$ sed -e 's/Will/Field/g' -e 's/Will/Jones/g' <example.txt              

Field reads well, Field writes well,Field sing well.

[gz_fieldyang@ test ~]$ echo Will reads well, Jones writes well,Kin sing well. >example.txt         

[gz_fieldyang@ test ~]$ sed -e 's/Will/Field/g' -e 's/Will/Jones/g' <example.txt          

Field reads well, Jones writes well,Kin sing well.

[gz_fieldyang@ test ~]$ cat example.txt

Will reads well, Jones writes well,Kin sing well.

[gz_fieldyang@ test ~]$ sed -e 's/Will/Field/g' -e 's/Jones/Jimmy/g' -e 's/Kin/Xie/g' <example.txt                         

Field reads well, Jimmy writes well,Xie sing well.

[gz_fieldyang@ test ~]$ sed -e 's/Will/Field/g' -e 's/Jones/Jimmy/g' -e 's/Kin/Xie/g' <example.txt >example2.txt

[gz_fieldyang@ test ~]$ cat example2.txt

Field reads well, Jimmy writes well,Xie sing well.

                                #使用sed -e 选项给予sed多个命令

ⅱ.sed -f ：自脚本中读取编辑命令

[gz_fieldyang@ test ~]$ cat > fixsed.sed                #将多项编辑命令放进脚本中

s/Will/\&\/etc\/example\/zhuangyizufu/

s/Will/\&&/

s/Will/Field/g

s/Jones/Jimmy/g

s/Kin/Xie/g

Ctrl+D                       #保存退出

[gz_fieldyang@ test ~]$

[gz_fieldyang@ test ~]$ echo Will reads well, Jones writes well,Kin sing well. >example.txt            #输入样本

[gz_fieldyang@ test ~]$ sed -f fixsed.sed example.txt 

      #sed -f :自脚本中读取编辑命令

&/etc/example/zhuangyizufu reads well, Jimmy writes well,Xie sing well.

[gz_fieldyang@ test ~]$ sed -f fixsed.sed example.txt >example2.txt

#也可重定向到新文件

[gz_fieldyang@ test ~]$ cat example2.txt

&/etc/example/zhuangyizufu reads well, Jimmy writes well,Xie sing well.

 

练习：创建一个html_to_xhtml的简单脚本，可以将HTML转换为XHTML

$ cat > html_to_xhtml.sed

S/<H1>/<h1>/g

S/<H2>/<h2>/g                 #/为定界符

S/<H3>/<h3>/g

S/<H4>/<h4>/g

S/<H5>/<h5>/g

S/<H6>/<h6>/g

s:</H1>:</h1>:g                      #：为定界符

s:</H2>:</h2>:g

s:</H3>:</h3>:g

s:</H4>:</h4>:g

s:</H5>:</h5>:g

s:</H6>:</h6>:g

s:</[Hh][Tt][Mm][LL]>:<html>:g

s:</[Hh][Tt][Mm][Ll]>:</html>:g

s:<[Bb][Rr]>:<br/>:g

...

Ctrl+D               #保存退出

sed -f html_to_xhtml.sed html.html > xhtml.xhtml

# 该脚本会将标签转换为小写，然后更改<br>标签为自我结束形式<br/>

 

④打印与否

sed -n 选项修改sed的默认行为，当提供该选项时不打印

反之，若在脚本里使用p，则会明白的将此项显示出来

此外，sed注释必须出现在单独的行里，因为他们是语法型命令，意思是什么也不做的命令

 

实例1：

[gz_fieldyang@ test ~]$ cat >test.html

<HTML>

<H1>                               

HELLO Field

</H1>

<HTML>TEXT HTML,NO USEFUL

</HTML>

[gz_fieldyang@ test ~]$ sed -n '/<HTML>/p' *.html             #仅显示<HTML>的行

<HTML>

<HTML>TEXT HTML,NO USEFUL

[gz_fieldyang@ test ~]$ cat > sed-n.sed

#n                 #sed注释必须出现在单独的行里，因为他们是语法型命令，意思是什么也不做的命令

/<HTML>/p

[gz_fieldyang@ test ~]$ sed -f sed-n.sed *

/<HTML>/p

<HTML>

<HTML>TEXT HTML,NO USEFUL

[gz_fieldyang@ test ~]$ sed -f sed-n.sed *.html

<HTML>

<HTML>TEXT HTML,NO USEFUL

[gz_fieldyang@ test ~]$

 

实例2：

[gz_fieldyang@ test ~]$ cat > sed-n.sed

/Field_yang/p                                # 注意，此处sed未加#n注释

/Jones_yan/p

[gz_fieldyang@ test ~]$ cat >test.txt

Field_yang ,how are you 

Kin_ma,you are well.

Jimmy_huang,nice to mean you

Jones_yan,long time no see.

[gz_fieldyang@ test ~]$ sed -f sed-n.sed *.txt

Field_yang ,how are you                       #都会被打印

Field_yang ,how are you

Kin_ma,you are well.

Jimmy_huang,nice to mean you

Jones_yan,long time no see.

Jones_yan,long time no see.

[gz_fieldyang@ test ~]$ cat > sed-n.sed          

#n                                           #此处添加#n

/Kin/p 

/Fie/p

[gz_fieldyang@ test ~]$ sed -f sed-n.sed test.txt

Field_yang ,how are you               # 只打印匹配的行

Kin_ma,you are well.

[gz_fieldyang@ test ~]$

 

⑤匹配特定行

[gz_fieldyang@ test ~]$ cat test.txt                   #创建文本                                

Field_yang ,how are you?

Kin_ma,you are well.

Jimmy_huang,nice to mean you

Jones_yan,long time no see.

Tracy_luo,hello.

Will_wu,hello.

This just a test ,

which for sed,

no useful.

符号$至最后一行 下列命令为快速打印最后一行

[gz_fieldyang@ test ~]$ sed -n '$p'"$1" test.txt                                                                                                     #匹配最后一行 ，引号里为指定显示的数据

no useful.               

[gz_fieldyang@ test ~]$ sed -n '$p' test.txt                    #匹配最后一行

no useful.

[gz_fieldyang@ test ~]$ sed -n "$1" test.txt    

打印特定的行 sed -n 'n,mp' file

[gz_fieldyang@ test ~]$ sed -n '5p' test.txt                   #打印第五行   

Tracy_luo,hello.

[gz_fieldyang@ test ~]$        

[gz_fieldyang@ test ~]$ sed -n '2,5p' test.txt               #打印二到五行                   

Kin_ma,you are well.

Jimmy_huang,nice to mean you

Jones_yan,long time no see.

Tracy_luo,hello.

[gz_fieldyang@ test ~]$ sed '/Jimmy/,/This/ s/Jones_yan/Anges_wen/g' test.txt        

Field_yang ,how are you?

Kin_ma,you are well.

Jimmy_huang,nice to mean you

Anges_wen,long time no see.

Tracy_luo,hello.

Will_wu,hello.

This just a test ,

which for sed,

no useful.

# 该命令的意思为“从含有Jimmy的行开始，到匹配到含This的行为止，将所有匹配的行中Jones_yan 全换为Anges_wen

[gz_fieldyang@ test ~]$ sed -e  '/luo/,/no/  s/hello/so good/g'  -e  '/Jimmy/,/This/  s/Jones_yan/Anges_wen/g' test.txt       

Field_yang ,how are you?

Kin_ma,you are well.

Jimmy_huang,nice to mean you

Anges_wen,long time no see.

Tracy_luo,so good.

Will_wu,so good.

This just a test ,

which for sed,

no useful.

[gz_fieldyang@ test ~]$

 

⑥正则表达式：

s命令里的空模式(//)表示”使用前一个正则表达式“

&在替代文本中表示的意思是“从此点开始替代成匹配于正则表达式的整个文本”

[gz_fieldyang@ test ~]$  echo Will reads well, Will writes well,Will sing well. >example.txt

[gz_fieldyang@ test ~]$  sed  '/Will/ s//& and Ken/g' example.txt                                              

Will and Ken reads well, Will and Ken writes well,Will and Ken sing well.

[gz_fieldyang@ test ~]$  sed  '/Will/ s//& and Ken/g' test.txt  

Field_yang ,how are you?

Kin_ma,you are well.

Jimmy_huang,nice to mean you

Jones_yan,long time no see.

Tracy_luo,hello.

Will and Ken_wu,hello.

This just a test ,

which for sed,

no useful.

[gz_fieldyang@ test ~]$  sed  '/Will_wu/ s//& and Ken_can/g' test.txt

Field_yang ,how are you?

Kin_ma,you are well.

Jimmy_huang,nice to mean you

Jones_yan,long time no see.

Tracy_luo,hello.

Will_wu and Ken_can,hello.

This just a test ,

which for sed,

no useful.

 

⑦否定正则表达式

通过加！在正则表达式后面即可实现

[gz_fieldyang@ test ~]$  echo Will reads well, Will writes well,Will sing well. >example.txt

[gz_fieldyang@ test ~]$  sed '/is/ !s/ill/entes/g ' example.txt 

                        # 将没有is的每个行里所有的ill改成entes                       

Wentes reads well, Wentes writes well,Wentes sing well.

[gz_fieldyang@ test ~]$

[gz_fieldyang@ test ~]$  sed '/is/ !s/are/is/g ' test.txt

Field_yang ,how is you?             # 将没有is的每个行里所有的are改成is

Kin_ma,you is well.

Jimmy_huang,nice to mean you

Jones_yan,long time no see.

Tracy_luo,hello.

Will_wu,hello.

This just a test ,

which for sed,

no useful.

 

实例：

如何在模式内使用不同的定界符，本例以冒号,-号,?号隔开要查找的模式，以分号,^号作为定界符

[gz_fieldyang@ test ~]$ grep gz_fieldyang /etc/passwd

gz_fieldyang:x:859:859::/home/gz_fieldyang:/bin/bash

[gz_fieldyang@ test ~]$ sed -n '\:gz_fieldyang: s;;GZ_FieldYang;p' /etc/passwd

GZ_FieldYang:x:859:859::/home/gz_fieldyang:/bin/bash

[gz_fieldyang@ test ~]$

[gz_fieldyang@ test ~]$ sed -n '\-gz_fieldyang- s;;GZ_FieldYang;p' /etc/passwd  

GZ_FieldYang:x:859:859::/home/gz_fieldyang:/bin/bash

[gz_fieldyang@ test ~]$ sed -n '\#gz_fieldyang# s^^GZ_FieldYang^p' /etc/passwd  

GZ_FieldYang:x:859:859::/home/gz_fieldyang:/bin/bash

[gz_fieldyang@ test ~]$

 

 

⑧匹配多少文本

[gz_fieldyang@ test ~]$ echo Jones-yan is good. | sed 's/J.*n/Field_yang/'

Field_yang is good.

[gz_fieldyang@ test ~]$ echo Jones is good. | sed 's/J[[:alpha:]]*s/Field_yang/'     

Field_yang is good.

[gz_fieldyang@ test ~]$ echo abc| sed 's/b*/1/'

1abc

#替代第一个匹配成功的，b*是如何匹配在abc的请与结尾的null字符串，*匹配0个或多个

[gz_fieldyang@ test ~]$ echo abc| sed 's/b*/1/g'                 #替代所有匹配成功的

1a1c1

 

3）.行与字符串法人区别

①大部分简易程序都是处理输入数据的行，像几乎所有的grep,egrep,以及sed，

这种情况不会有内嵌换行字符出现在将要匹配的数据中，则^$分别表示行的开头结尾

②对可应用于正则表达式的程序语言，例如awk,perl,python所处理的基本为字符串

若每个字符串表示的就是独立的一行输入，则^$仍可分别表示行的开始结尾，

如果行中内嵌换行字符，则^$无法匹配内嵌的换号字符，它们只用来表示字符串的开头和结尾。