shell编程之行编辑器 - sed 对文件进行操作

行编辑器 - sed 介绍

sed 介绍：

在shell脚本编写中，时常会对文件进行相关操作，比如对文件内容进行增删改查。但上述操作一般需要在文本编辑器中才能操作。这又违背了我们编写shell脚本的意愿（自动化，减少人为操作，让机器来完成），这就需要sed命令了

sed是Linux中提供的一个外部命令，它是一个行（流）编辑器。区别于vim、gedit、nano等常用交互式文本编辑器。sed行编辑器的可以非交互式的对文件内容进行增删改查的操作，使用者只能在命令行中输入编辑命令、指定文件名，然后在屏幕上查看输出

• 文本编辑器：编辑的对象是文件

• 行编辑器：编辑的对象是文件中的行（内容）

也就是说文本编辑器一次处理一个文件；而行编辑器一次处理一个文本中的一行

sed 处理数据原理：

• sed：行编辑器，在内存中编辑，然后输出到屏幕
  

sed 语法

• sed默认读取'{command}[flags]'单引号内第一个字符当命令，且默认对所有行操作（可以在命令后加\转义字符区分命令与操作字符）

• 命令前编写，如：'1命令' => 表示在第一行进行操作；'2,4命令' => 表示在2至4行进行操作

• command：可以用-e选项配合分号;分割多个命令可执行多个操作

匹配模式(/匹配内容/)：在命令前增加/匹配内容/，如'/^2 This/command}[flag]'，表示匹配以2 This开头的行对其进行操作

sed 语法：

语法格式：
	# sed [options] '{command}[flags]' [filename]
		options说明：
			-e script：将脚本中指定的命令添加到处理输入时执行的命令中，多条件，一行要有多个操作
			-f scripi：将文本中指定的命令添加到处理输入时的命令中
			-n：抑制内存自动输出
			-i：编辑文件内容
			-i.bak：修改时同时创建.bak备份文件
			-r：使用扩展的正则表达式。因为需要输出，所以需要和-p选项配合使用（还可以使用-n选项一致输出，免得重复输出内容）
			!：取反（跟在模式条件后与shell有所区别）


		command说明：
			增：
				a：在匹配行"后"新开一行
				i：在匹配行"前"新开一行
			删：
				d：删除。删除指定行
			改：
				s：查找替换。替换指定行的字符串
				c：更改。更改的是行内容
				y：转换。转换的是行内容
			查：
				p：打印。打印两边：1遍是文件内容，1遍是内存输出内容


		flags说明：
			数字：表示新文本替换的模式，与s命令配合表示替换第几个匹配的字符串
			g：表示新文本替换现有文本的全部示例（与s命令配合，因为s命令只替换每行第一个匹配字符串）
			p：表示打印原始的内容（配合s命令使用）
			w dilename：将替换的结果写入文件（与s命令配合使用），不影响原文件内容

#l	注意：修改输出的是内存中的内容，

command 命令说明

增 演示：a 后、i 前

• $a命令表示在最后一行后面增加一行
• $i命令表示在最后一行前面增加一行

[root@server ~]# cat sed.txt
	1 This little sister is my dish
	2 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish

#l	a：在行后增加		=>	'a增加内容'
[root@server ~]# sed 'a\heloo,这是我的菜' sed.txt		=>	不指定行默认所有
	1 This little sister is my dish
	heloo,这是我的菜
	2 This little sister is my dish
	heloo,这是我的菜
	3 This little sister is my dish
	heloo,这是我的菜
	4 This little sister is my dish
	heloo,这是我的菜

[root@server ~]#  sed '$aheloo,这是我的菜' sed.txt	=>	"$a" 在最后一行，增加a命令后内容
1 This little sister is my dish
2 This little sister is my dish
3 This little sister is my dish
4 This little sister is my dish
heloo,这是我的菜


[root@server ~]# sed '2aheloo,这是我的菜' sed.txt	=>	"2a" 只有第二行后
	1 This little sister is my dish
	2 This little sister is my dish
	heloo,这是我的菜
	3 This little sister is my dish
	4 This little sister is my dish

[root@server ~]# sed '2,4a\heloo,这是我的菜' sed.txt		=>	"2,4a" 2至4行后
	1 This little sister is my dish
	2 This little sister is my dish
	heloo,这是我的菜
	3 This little sister is my dish
	heloo,这是我的菜
	4 This little sister is my dish
	heloo,这是我的菜
#l 匹配
[root@server ~]# sed '/^2 Thi/aheloo,这是我的菜' sed.txt		=>	"/^2 Thi/a" 以2 Thi开头的行后
	1 This little sister is my dish
	2 This little sister is my dish
	heloo,这是我的菜
	3 This little sister is my dish
	4 This little sister is my dish



#l	i：在行前增加		=>	'i增加内容'
[root@server ~]# sed 'iheloo,这是我的菜' sed.txt		=>	"i" 所有行前面，增加i命令后内容
		heloo,这是我的菜
		1 This little sister is my dish
		heloo,这是我的菜
		2 This little sister is my dish
		heloo,这是我的菜
		3 This little sister is my dish
		heloo,这是我的菜
		4 This little sister is my dish
	#l	同理，i 也可以用$i 2i  2,4i 或匹配 /^1 T/i  等进行操作		

删 演示：d 删

• s命令如果是替换成空不需要加空格，直接s/要替换字符串//就行

[root@server ~]# cat sed.txt
	1 This little sister is my dish
	2 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish
	5 This little sister is my dish
	6 This little sister is my dish
	7 This little sister is my dish
	8 This little sister is my dish

#l	d：删除		=>	'd' 删除行
[root@server ~]# sed '3d' sed.txt		=>	"3d" 只删第3行
	1 This little sister is my dish
	2 This little sister is my dish
	4 This little sister is my dish
	5 This little sister is my dish
	6 This little sister is my dish
	7 This little sister is my dish
	8 This little sister is my dish

[root@server ~]# sed '1,6d' sed.txt		=>	"1,6d" 删除文件第1至第6行
	7 This little sister is my dish
	8 This little sister is my dish
[root@server ~]# sed 'd' sed.txt		=>	"d" 删除文件所有行（即删除文件内容）

#l 匹配
[root@server ~]# sed '/^1 T/d' sed.txt	=>	"/^1 T/d" 删除以/^1 T/开头的内容的内容
	2 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish
	5 This little sister is my dish
	6 This little sister is my dish
	7 This little sister is my dish
	8 This little sister is my dish

改 演示：s 替换、c 更改、y 转换

• s命令单用只更改每行中替换与被替换匹配的第1个字符串
• c命令更改的是整行
• y命令转换的也是整体文件，匹配则转换
• 删除＃号 ，用s命令将#替换成空's/#//' （注意：要替换的地方没有空格）
• 增加#号 ，用命令将空替换成#‘s/ /#’（注意：被替换的地方有空格）

[root@server ~]# cat sed.txt
	1 This little sister is my dish
	2 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish


#l	s：替换		=>	's/被替换字符串/要替换字符串/'
	[root@server ~]# sed 's/my/you/' sed.txt	=>	"s" 将所有行中的第一个my替换成you
		1 This little sister is you dish
		2 This little sister is you dish
		3 This little sister is you dish
		4 This little sister is you dish
	#l	同理，s 也可以用 2s  2,4s 或匹配 /^1 T/s  等进行操作
		
		

#l	c：更改		=>	'c更改内容'
	[root@server ~]# sed 'c不要迷恋网游' sed.txt		=>	"c" 将所有行更改
		不要迷恋网游
		不要迷恋网游
		不要迷恋网游
		不要迷恋网游
		
	[root@server ~]# sed '2,3c不要迷恋网游' sed.txt		=>	"2,3c" 将2至3行的内容变成一行
		1 This little sister is my dish
		不要迷恋网游
		4 This little sister is my dish		
	#l	同理，c 也可以用 2c 或匹配 /^1 T/c  等进行操作



#l	y：转换		=>	'y/被转换的字符/转换的字符/' （注意：被转换字符和转换字符一一对应）
	[root@server ~]# sed 'y/my/MY/' sed.txt		=>	my转换成MY
		1 This little sister is MY dish
		2 This little sister is MY dish
		3 This little sister is MY dish
		4 This little sister is MY dish

	[root@server ~]# sed 'y/my/YM/' sed.txt		=>	my转换成YM
		1 This little sister is YM dish
		2 This little sister is YM dish
		3 This little sister is YM dish
		4 This little sister is YM dish
	#l	同理，y 也可以用 2y  2,4y 或匹配 /^1 T/y  等进行操作				

查 演示：p 打印（两遍）

• 是将原文件内容打印一遍，内存内容打印一遍

• 需要和标志p 区分开，标志p打印的是修改过的行；p命令则是原原本本全部打印一遍原文件内容

[root@server ~]# cat sed.txt
	1 This little sister is my dish
	2 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish


#l	p：打印		=>	'd'	打印文件内容
[root@server ~]# sed 'p' sed.txt
	1 This little sister is my dish
	1 This little sister is my dish
	2 This little sister is my dish
	2 This little sister is my dish
	3 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish
	4 This little sister is my dish


#l	内存内容被修改，原文件的内容没有被修改
[root@server ~]# sed -e 'p;s/my/MMMMMM/' sed.txt
	 1 This little sister is my dish
	 1 This little sister is MMMMMM dish
	 2 This little sister is my dish
	 2 This little sister is MMMMMM dish
	 3 This little sister is my dish
	 3 This little sister is MMMMMM dish
	 4 This little sister is my dish
	 4 This little sister is MMMMMM dish

#l	打印两次：打印文件输出一遍，同时sed命令把文件输入内存又打印一遍

options 选项说明

-e 演示：多条件用 ; 分割

• 当需要一行有多个操作时，需要用-e指定多个条件
• ;分号分割多个条件

[root@server ~]# cat sed.txt
	1 This little sister is my dish
	2 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish


[root@server ~]# sed -e 's/little/big/;s/my/you/' sed.txt	=>	"s"	将little和my替换成big和you
	1 This big sister is you dish
	2 This big sister is you dish
	3 This big sister is you dish
	4 This big sister is you dish
[root@server ~]# sed -e 's/little/big/;4c我还要改第四行内容' sed.txt		=>	"s和c"	两个命令不分先后
	1 This big sister is my dish
	2 This big sister is my dish
	3 This big sister is my dish
	我还要改第四行内容

-f 演示：调用文本命令

• 将需要执行的sed命令放入文本中，需要执行时用-f指定调用

[root@server ~]# cat sed.txt
1 This little sister is my dish
2 This little sister is my dish
3 This little sister is my dish
4 This little sister is my dish


#l 文本中的命令：
	[root@server ~]# vim test
		1a我在第一个行后面
		s/sister/big brother/
		4i我在第4行前面


#l	-f：执行文本命令
	[root@server ~]# sed -f text sed.txt
		1 This little big brother is my dish
		我在第一个行后面
		2 This little big brother is my dish
		3 This little big brother is my dish
		我在第4行前面
		4 This little big brother is my dish

-n 演示：抑制内存输出

• 抑制内存输出，只输出文件内容

[root@server ~]# cat sed.txt
1 This little sister is my dish
2 This little sister is my dish
3 This little sister is my dish
4 This little sister is my dish



#l	-n：p命令打印时，抑制内存输出
[root@server ~]# sed -n "p" sed.txt		
	1 This little sister is my dish
	2 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish

-i 演示：更改原文件

• 加-i选项改的是文件内容，不加-i选项改的是内存内容
• 不可逆操作，改了就不能撤销，慎用

-i选项和-i.bak选项操作一样

····
····
····

-i.bak 演示：先备份再更改原文件

• 加-i.bak改的是文件内容，不加-i.bak改的是内存内容
• -i.bak会先对原文件进行备份成.bak文件，然后再对文件进行修改，推荐使用

[root@server ~]# cat sed.txt
1 This little sister is my dish
2 This little sister is my dish
3 This little sister is my dish
4 This little sister is my dish


#l -i.bak：创建备份文件，修改原文件内容
	[root@server ~]# cat text	=>	-f 的命令文本
		1a我在第一个行后面
		s/sister/big brother/
	[root@server ~]# sed -i.bak -f text sed.txt		=>	执行"-i.bak"	
	[root@server ~]# cat sed.txt				=>	查看原文件是否更改
		1 This little big brother is my dish
		我在第一个行后面
		2 This little big brother is my dish
		3 This little big brother is my dish
		我在第4行前面
		4 This little big brother is my dish

	[root@server ~]# ll sed.txt.bak		=>	查看是否生成.bak备份文件
		-rw-r--r-- 1 root root 129 Jun  4 06:01 sed.txt.bak
	[root@server ~]# cat sed.txt.bak		=>	查看备份文件内容是否和原始文件内容一样
		1 This little sister is my dish
		2 This little sister is my dish
		3 This little sister is my dish
		4 This little sister is my dish

-r 演示：扩展正则

• 支持扩展的正则表达式，功能更强
• 因为需要查看输出，所以可以配合-p选项（打印）和-n选项（抑制内存输出）
• 匹配符为/正则表达式/

#l	-r：扩展正则
	[root@server ~]# sed -n -r '/^(a|b)c/p' sed1
		acb
	[root@server ~]# sed -n -r '/^a.*c/p' sed1
		abc
		acb
		abbc

flags 标志说明

数字 演示：匹配第几个替换字符串

• 配合s命令使用，表示替换行中第几个被匹配到的替换字符串

[root@server ~]# cat sed2
	you can you up you just say
	you can you up you just say
	you can you up you just say
	you can you up you just say
	you can you up you just say


#l	单s命令替换：
	[root@server ~]# sed 's/you/I/' sed2		=>	只替换第一个匹配字符串
		I can you up you just say
		I can you up you just say
		I can you up you just say
		I can you up you just say
		I can you up you just say
		
#l	flags数字与s命令配合使用：
		[root@server ~]# sed 's/you/I/2' sed2		=>	数字为2表示替换s命令中每行第二个匹配的字符串
			you can I up you just say
			you can I up you just say
			you can I up you just say
			you can I up you just say
			you can I up you just say
	#l	以此类推，数字3表示替换每行第3个匹配字符串

g 演示：替换所有

• 与s命令配合使用表示将文件中所有被匹配到的全部转换

[root@server ~]# cat sed2
	you can you up you just say
	you can you up you just say
	you can you up you just say
	you can you up you just say
	you can you up you just say


#l	标志g与s命令配合使用：替换所有匹配的示例
		[root@server ~]# sed 's/you/I/g' sed2
			I can I up I just say
			I can I up I just say
			I can I up I just say
			I can I up I just say
			I can I up I just say

w filename 演示：将替换结果到文件中

• 将替换结果保存到文件中
• w filename：filename 可以是一个路径
• 保存的内容是更改的内容

[root@server ~]# cat sed2
	you can you up you just say
	you can you up you just say
	you can you up you just say
	you can you up you just say
	you can you up you just say


#l	标志 w filename：保存路径默认是执行命令的当前目录下
		[root@server ~]# sed  '2s/can/ooo/w replace_sed' sed2	=>	执行
			you can you up you just say
			you ooo you up you just say
			you can you up you just say
			you can you up you just say
			you can you up you just say
		[root@server ~]# cat replace_sed		=>	查看文件是否生成，并查看内容
			you ooo you up you just say

p 演示：打印原始内容（被修改过的内容）

• 打印原始文件内容，注意打印的是awk修改后的内容。

• 相当于awk修改过的行重新打印一遍，没修改的不打印

• 注意：p命令和标志p需要分清楚，p命令是打印的是原原本本的文件内容和内存内容；而标志p打印的是被修改后的原始内容和内存内容

• 强行这样解释，标志p打印出来的就是原文件被修改过的内容

[root@server ~]# cat sed.txt
	1 This little sister is my dish
	2 This little sister is my dish
	3 This little sister is my dish
	4 This little sister is my dish


#l	标志 p：输出原始文件内容
	[root@server ~]# sed '1s/This/that/p' sed.txt	=>	标志p输出的原始内容会打印awk修改过的行加内存输出的内容
		1 that little sister is my dish
		1 that little sister is my dish
		2 This little sister is my dish
		3 This little sister is my dish
		4 This little sister is my dish

	[root@server ~]# sed 'p;1s/This/that/' sed.txt	=>	p命令会输出原文件内容，打印两遍，但只有内存输出的信息更改了，所以只更改了一条数据
		1 This little sister is my dish
		1 that little sister is my dish
		2 This little sister is my dish
		2 This little sister is my dish
		3 This little sister is my dish
		3 This little sister is my dish
		4 This little sister is my dish
		4 This little sister is my dish


	[root@server ~]# sed -n 'p;1s/This/that/' sed.txt	=>	p命令打印两遍，但是-n抑制内存输出，所以打印的是原始文件，原始文件的内容是没有更改的
		1 This little sister is my dish
		2 This little sister is my dish
		3 This little sister is my dish
		4 This little sister is my dish
	[root@server ~]# sed -n '1s/This/that/p' sed.txt	=>	抑制了内存输出，那么打印出来的就是被修改后的原始文件内容
		1 that little sister is my dish