sed详解乞丐版

简介

sed(Stream EDiter)作为linux文本处理三剑客之一，是一个面向行的文本处理工具（行编辑器）。

工作模式

sed从一个输入流或者文件中逐行读取文本到一个名为模式空间(pattern space)的内部缓冲区中。在模式空间中，sed可以通过一个sed脚本对文本进行一个或者多个特定的操作。对于每个输入行，默认情况下，sed会先复制该行内容到模式空间，在运行完sed脚本之后，sed会输出模式空间的内容，然后再读取下一行。所以，sed默认是不会修改原文件的。如果需要直接修改原文件，需要加特定选项(-i)。
sed脚本可以在命令中被指定（使用-e选项），也可以从一个独立文件中读取（使用-f选项）。除了模式空间，sed还有另一个独立的缓冲区，名为保持空间（hold space），用来保持和累计文本。不过大多数情况下，我们所要完成的操作都在模式空间中完成。
上面提到的sed脚本不同于我们常见的shell脚本，它并不是常规意义上的一个文件，而是一个模式-动作对（也可以理解为“地址定界+编辑命令”），脚本指示出了要匹配的模式和要执行的操作：例如"2d"，代表删除第二行。

sed的用法

• sed的语法格式：
  sed [OPTION]…  {script}  [input-file]…
  sed [OPTION]…  -f  scriptfile   [input-file]…

常用选项(OPTION)：

OPTION	长格式OPTION	说明
-n	- -quiet, - -silent	仅显示script处理后的结果，即不再默认打印模式空间中的内容。
-e	script, - -expression=script	以指定sed脚本进行文本处理，可实现多点编辑
-f script-file	- -file=script-file	从文件读取sed脚本，每行是一个独立的sed命令
-r	- -regexp-extended	支持使用扩展正则表达式；
-i[SUFFIX]	, - -in-place[=SUFFIX]	直接编辑原文件 ；
	- -help	显示帮助
	- -version	显示sed版本信息

脚本（SCRIPT）:

地址定界：

定界方式	说明
默认,空地址	对全文进行
m	第m行
m,n	从m到n的连续行
m,+n	从m行到m+n行（共n+1行）
/pattern/	被此模式匹配到的每一行
m,/pattern/	第m行开始到第一次被pattern匹配到的行结束
/pattern1/,/pattern2/	从第一次被pattern1匹配到的行开始，到第一次被pattern2匹配到的行结束
~	步进，例如1~2，代表所有奇数行
$	最后一行

编辑命令：

编辑命令	说明
p	显示模式空间中的内容
d	删除模式空间中的行
a \text	在行后面追加文本“text”，支持使用\n实现多行追加
c \text	把匹配到的行替换为此处指定的文本“text”
i \text	在行前面插入文本“text”，支持使用\n实现多行插入
s	查找替换，其分隔符可自行指定，常用的有s///,s@@@, s###等
r file	读取指定文件的内容至当前文件被模式匹配到的行后面；可实现文件合并
w file	保存模式空间匹配到的行至指定的文件中（会覆盖指定文件的内容）
!	条件取反,格式为：地址定界!编辑命令，表示编辑命令对没有被地址定界匹配到的行发生作用，对被匹配到的行反而不起作用
=	为模式匹配到的行打印行号，注意是在匹配到的行的前一行，单起一行打印行号
h	把模式空间中的内容覆盖至保持空间中
H	把模式空间中的内容追加至保持空间中
g	把保持空间中的内容覆盖至模式空间中
G	把保持空间中的内容追加至模式空间中
x	把模式空间中的内容与保持空间中的内容互换
l	完整显示行内容（即行尾添加行结束符）
n	覆盖读取匹配到的行的下一行至模式空间中
N	追加读取匹配到的行的下一行至模式空间中
q	退出sed

替换命令标记

上面提到的编辑命令中的查找替换命令(s)，还有一些标记可选。

替换标记	说明
g	行内全局替换
p	显示替换成功的行
w	将替换成功的行保存至指定文件中（会覆盖指定文件中的内容）
x	表示互换模式空间和保持空间中的内容

实例讲解

测试文件(sante fe的一段歌词)

[root@centos7u6 ~]# cat sante_fe.txt 
They say that no man is an island
And good things come to those who wait
But the things I hear are there
Just to remind me
Every dog will have his day
#
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

• 使用cat -e -E选项显示出文本的行数和行结束符，可以注意到第6行为一个#，第7行为空格（空白字符），第12行为空白行。

[root@centos7u6 ~]# cat -n -E sante_fe.txt 
     1  They say that no man is an island$
     2  And good things come to those who wait$
     3  But the things I hear are there$
     4  Just to remind me$
     5  Every dog will have his day$
     6  #$
     7   $
     8  And I blame this world for making a good man evil$
     9  It's this world that can drive a good man mad$
    10  And it's this world that turns a killer into a hero$
    11  Well I blame this world for making a good man bad$
    12  $
[root@centos7u6 ~]# 

重申sed工作模式(很重要)

sed的工作模式大致是这样的：一段文本流的一行被读入到模式空间中后，会被命令所给的“模式-行为对”进行匹配：

• 如果这行文本没有被匹配到，sed默认将模式空间中的内容（该行的内容）显示到标准输出。
• 如果被匹配到， sed会将匹配到的行进行特定操作，操作完成后，默认再将模式空间中的内容（此时的内容不一定是原来该行的内容了）显示至标准输出。
• 看上去可能有点绕，接下来我们先拿两个相对奇葩的编辑命令来讲解一下。

p命令

[root@centos7u6 ~]# sed '2p' sante_fe.txt 
They say that no man is an island
And good things come to those who wait
And good things come to those who wait
But the things I hear are there
Just to remind me
Every dog will have his day
#
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

[root@centos7u6 ~]# 

• 是不是想要的结果和实际显示结果不一样。。。。
• 这里来分析一下，sed脚本内容是打印第2行，由于sed默认会将没有被匹配到的行打印至屏幕，那么其他行显示出来就不足为奇了。
• 现在要说一说为什么第2行显示了2次，是因为当第2行被读入空间后，由于编辑命令是p（打印），所以执行特定操作即打印第2行的内容；而 后，sed默认打印模式空间中的行，此时第2行的内容没有改变，所以，我们看到第2行打印了2次。
• 如果只是想打印第2行内容，应加上 -n 选项：

[root@centos7u6 ~]# sed -n '2p' sante_fe.txt 
And good things come to those who wait

d命令

[root@centos7u6 ~]# sed '3,12d' sante_fe.txt 
They say that no man is an island
And good things come to those who wait

• 理解了上面的p命令，那么d命令就好理解多了，上述命令的结果和sed脚本的字面意义上的结果是一样的，那么我们来看一下d命令的工作流程。
• 首先，还是一样的，没有被匹配到的行，默认打印出来。（1,2行）
• 从第3行开始，被匹配到，由于对行的特定操作是删除，所以特定操作完成后，模式空间中的内容变为空，sed默认打印模式空间中的内容也是空，结果就是在标准输出上，我们看不到被删除的行了。
• 除了这两个命令外，其他命令都是相对正常的了。。。

i命令

[root@centos7u6 ~]# sed  '6i \WTF\nOMFG' sante_fe.txt 
They say that no man is an island
And good things come to those who wait
But the things I hear are there
Just to remind me
Every dog will have his day
WTF
OMFG
#
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

• 在第6行前插入2行内容，其中，\n为换行符

a命令

[root@centos7u6 ~]# sed '6a \WTF\n\OMFG' sante_fe.txt 
They say that no man is an island
And good things come to those who wait
But the things I hear are there
Just to remind me
Every dog will have his day
#
WTF
OMFG
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

• 在第6行后插入2行内容，其中，\n为换行符

c命令

[root@centos7u6 ~]# sed '6c \WTF\n\OMFG' sante_fe.txt 
They say that no man is an island
And good things come to those who wait
But the things I hear are there
Just to remind me
Every dog will have his day
WTF
OMFG
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

[root@centos7u6 ~]

s命令

默认

[root@centos7u6 ~]# sed 's/a/A/' sante_fe.txt 
They sAy that no man is an island
And good things come to those who wAit
But the things I heAr are there
Just to remind me
Every dog will hAve his day
#
 
And I blAme this world for making a good man evil
It's this world thAt can drive a good man mad
And it's this world thAt turns a killer into a hero
Well I blAme this world for making a good man bad

• 空地址（不加地址定界），表示所有行都进行替换
• 空标记（后面不加替换标记），表示替换每行中第一个被匹配到的字段，后面的不再替换

g替换标记

[root@centos7u6 ~]# sed 's/a/A/g' sante_fe.txt 
They sAy thAt no mAn is An islAnd
And good things come to those who wAit
But the things I heAr Are there
Just to remind me
Every dog will hAve his dAy
#
 
And I blAme this world for mAking A good mAn evil
It's this world thAt cAn drive A good mAn mAd
And it's this world thAt turns A killer into A hero
Well I blAme this world for mAking A good mAn bAd

• 命令加了g（行内全局标记），表示一行中所有被匹配到的字段都进行替换

加地址定界

[root@centos7u6 ~]# sed '1,3s/a/A/g' sante_fe.txt 
They sAy thAt no mAn is An islAnd
And good things come to those who wAit
But the things I heAr Are there
Just to remind me
Every dog will have his day
#
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

• 加了地址定界，只有定界匹配到的行进行替换操作

w标记(小写)

[root@centos7u6 ~]# sed '1,3s/a/A/w /root/123aToA.txt' sante_fe.txt  
They sAy that no man is an island
And good things come to those who wAit
But the things I heAr are there
Just to remind me
Every dog will have his day
#
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

[root@centos7u6 ~]# cat 123aToA.txt 
They sAy that no man is an island
And good things come to those who wAit
But the things I heAr are there

• 保存会覆盖文件（aToA.txt）的内容

r命令

[root@centos7u6 ~]# sed '6r 123aToA.txt' sante_fe.txt 
They say that no man is an island
And good things come to those who wait
But the things I hear are there
Just to remind me
Every dog will have his day
#
They sAy that no man is an island
And good things come to those who wAit
But the things I heAr are there
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

• 在第6行行后，抓入了aToA.txt（上面例子得到的文件）的内容

w命令

[root@centos7u6 ~]# sed '4,6s/e/E/g w  456eToE.txt' sante_fe.txt 
They say that no man is an island
And good things come to those who wait
But the things I hear are there
Just to rEmind mE
EvEry dog will havE his day
#
 
And I blame this world for making a good man evil
It's this world that can drive a good man mad
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

[root@centos7u6 ~]# cat 456eToE.txt 
Just to rEmind mE
EvEry dog will havE his day

• 将4到6行中，有替换操作的行保存至456eToE.txt文件中
• 第6行，没有发生替换操作，所以不保存
• 保存会覆盖456eeToE.txt中的原内容

!命令

[root@centos7u6 ~]# sed '1,3!d' sante_fe.txt 
They say that no man is an island
And good things come to those who wait
But the things I hear are there

• 1到3行不删，表示删除其他行，所以默认显示前3行

=命令

[root@centos7u6 ~]# sed   '/^And/=' sante_fe.txt 
They say that no man is an island
2
And good things come to those who wait
But the things I hear are there
Just to remind me
Every dog will have his day
#
 
8
And I blame this world for making a good man evil
It's this world that can drive a good man mad
10
And it's this world that turns a killer into a hero
Well I blame this world for making a good man bad

• 为所有以And开头的行打印行号

q命令

[root@centos7u6 ~]# sed   '/^And/q' sante_fe.txt 
They say that no man is an island
And good things come to those who wait

• 遇到And开头的行就退出sed命令，即不再向下读取文件内容

l命令

[root@centos7u6 ~]# sed  -n  '1l' sante_fe.txt 
They say that no man is an island$

• 完整显示第1行的内容

常用选项的应用

-e选项

[root@centos7u6 ~]# sed -n  -e 's/^[[:upper:]]*//' -e 'l'  sante_fe.txt 
hey say that no man is an island$
nd good things come to those who wait$
ut the things I hear are there$
ust to remind me$
very dog will have his day$
#$
 $
nd I blame this world for making a good man evil$
t's this world that can drive a good man mad$
nd it's this world that turns a killer into a hero$
ell I blame this world for making a good man bad$
$

• 上面命令为，删除所有以大写字母开头的行的首字母，并完整显示行的内容
• 因为这里是两步操作，所以使用-e选项，完成多点编辑。

-r选项

• 为了显示效果，先在sante_fe.txt文件末尾再追加一行，内容为3个空格

[root@centos7u6 ~]# echo '    ' >> sante_fe.txt 

• 默认（不支持扩展正则表达式）

[root@centos7u6 ~]# sed -n  -e 's/^[[:space:]]+/not blank/' -e 'l' sante_fe.txt 
They say that no man is an island$
And good things come to those who wait$
But the things I hear are there$
Just to remind me$
Every dog will have his day$
#$
 $
And I blame this world for making a good man evil$
It's this world that can drive a good man mad$
And it's this world that turns a killer into a hero$
Well I blame this world for making a good man bad$
$
    $
[root@centos7u6 ~]# sed -n  -e 's/^[[:space:]]\+/not blank/' -e 'l' sante_fe.txt 
They say that no man is an island$
And good things come to those who wait$
But the things I hear are there$
Just to remind me$
Every dog will have his day$
#$
not blank$
And I blame this world for making a good man evil$
It's this world that can drive a good man mad$
And it's this world that turns a killer into a hero$
Well I blame this world for making a good man bad$
$
not blank$
[root@centos7u6 ~]# 

• sed脚本的内容为将所有以空白字符开头的行的行首的所有空白字符替换为not blank
• sed默认只支持基本正则表达式，不支持扩展正则表达式，所以’+'号需要转义。

[root@centos7u6 ~]# sed  -r -n  -e 's/^[[:space:]]+/not blank/' -e 'l' sante_fe.txt 
They say that no man is an island$
And good things come to those who wait$
But the things I hear are there$
Just to remind me$
Every dog will have his day$
#$
not blank$
And I blame this world for making a good man evil$
It's this world that can drive a good man mad$
And it's this world that turns a killer into a hero$
Well I blame this world for making a good man bad$
$
not blank$

+加上-r选项后，sed支持扩展正则表达式，所以’+'号不再需要转义

暂时，就先写这些，后期如果总结出一些常用的sed脚本（再次注意：这里的脚本不是指文件，而是指上面那些单引号‘’括起来的地址定界+编辑命令对），会继续为大家分享出来。