一. 基本认识
sed处理文本的基本逻辑:
sed处理文本时是以行为单位的,处理时,把当前处理的行存储在临时缓冲区中,接着用sed 命令处理缓冲区中的内容,处理完成后,把缓冲区的内容送往屏幕。然后再处理下一行,直至全文处理结束。
sed可以:删除、查找替换、添加、插入、从其他文件中读入数据等。
常用场景是:
- 太过于庞大的文本,使用常规编辑器难以胜任(比如说vi一个几百兆的文件)。
- 有规律的文本修改,加快文本处理速度(比如说全文替换)。
常用选项:
-n:不输出模式空间内容到屏幕,即不自动打印,只打印匹配到的行
-e:多点编辑,对每行处理时,可以有多个Script
-f:把Script写到文件当中,在执行sed时-f 指定文件路径,如果是多个Script,换行写
-r:支持扩展的正则表达式
-i:直接将处理的结果写入文件
-i.bak:在将处理的结果写入文件之前备份一份
二. 常见用法
1. 多编辑:e
使用-e参数或分号连接多编辑命令
#两个编辑命令前都要使用-e参数,如果有更多的编辑需求,以此类推即可
[root@localhost ~]# sed -e 's/this/That/g' -e 's/line/LINE/g' Sed.txt
That is LINE 1,That is First LINE
That is LINE 2,the Second LINE,Empty LINE followed
That is LINE 4,That is Third LINE
That is LINE 5,That is Fifth LINE
#上面的命令可以用分号改写为:
[root@localhost ~]# sed 's/this/That/g ; s/line/LINE/g' Sed.txt
2. 删除:d
使用d命令可删除指定的行(源文件没有更改),示例如下:
#将file的第一行删除后输出到屏幕
[root@localhost ~]# sed '1d' Sed.txt
this is line 2,the Second line,Empty line followed
this is line 4,this is Third line
this is line 5,this is Fifth line
范围删除:
#删除指定范围的行(第1行到第3行)
[root@localhost ~]# sed '1,3d' Sed.txt
this is line 4,this is Third line
this is line 5,this is Fifth line
#删除指定范围的行(第一行到最后行)
[root@localhost ~]# sed '1,$d' Sed.txt
[root@localhost ~]# #清空了Sed.txt文件
其他特殊位置删除
#删除最后一行
[root@localhost ~]# sed '$d' Sed.txt
this is line 1,this is First line
this is line 2,the Second line,Empty line followed this is line 4,this is Third line
#删除除指定范围以外的行(只保留第5行)
#要删除其他的行请举一反三
[root@localhost ~]# sed '5!d' Sed.txt
this is line 5,this is Fifth line
#删除所有包含Empty的行
[root@localhost ~]# sed '/Empty/d' Sed.txt
this is line 1,this is First line
this is line 4,this is Third line
this is line 5,this is Fifth line
#删除空行
[root@localhost ~]# sed '/^$/d' Sed.txt
this is line 1,this is First line
this is line 2,the Second line,Empty line followed
this is line 4,this is Third line
this is line 5,this is Fifth line
3. 查找替换:s
使用s可将查找到的文本替换为新的文本。
替换开头
#只替换开头的this为that
[root@localhost ~]# sed 's/^this/that/' Sed.txt
that is line 1,this is First line
that is line 2,the Second line,Empty line followed
that is line 4,this is Third line
that is line 5,this is Fifth line
替换行内前几个
#s命令用于替换文本,本例中使用LINE替换line
#请注意每一行只有第一个line被替换了,默认情况下只替换第一次匹配到的内容
[root@localhost ~]# sed 's/line/LINE/' Sed.txt
this is LINE 1,this is First line
this is LINE 2,the Second line,Empty line followed
this is LINE 4,this is Third line
this is LINE 5,this is Fifth line
#要想每行最多匹配2个line,并改为LINE,可用如下方式
#注意到第2行中有3个line,前两个被替换了,第三个没有变化
[root@localhost ~]# sed 's/line/LINE/2' Sed.txt
this is line 1,this is First LINE
this is line 2,the Second LINE,Empty line followed
this is line 4,this is Third LINE
this is line 5,this is Fifth LINE
替换所有匹配
#s命令利用g选项,可以完成所有匹配值的替换
[root@localhost ~]# sed 's/line/LINE/g' Sed.txt
this is LINE 1,this is First LINE
this is LINE 2,the Second LINE,Empty LINE followed
this is LINE 4,this is Third LINE
this is LINE 5,this is Fifth LINE
4. 字符转换:y
y的逻辑是将一系列字符逐个地变换为另外一系列字符,基本用法如下
#该命令会将file中的O转换为N、L转换为E、D转换为W
#注意转换字符和被转换字符的长度要相等,否则sed无法执行
sed 'y/OLD/NEW/' file
[root@localhost ~]# sed 'y/1245/ABCD/' Sed.txt
this is line A,this is First line
this is line B,the Second line,Empty line followed
this is line C,this is Third line
this is line D,this is Fifth line
5. 插入文本:i、a
要想保存修改后的文件,必须使用重定向生成新的文件。如果想直接修改源文件本身则需要使用“-i”参数。
使用i或a命令插入文本,其中i代表在匹配行之前插入,而a代表在匹配行之后插入。
指定行插入文本
#使用i在第二行前插入文本
[root@localhost ~]# sed '2 i Insert' Sed.txt
this is line 1,this is First line
Insert
this is line 2,the Second line,Empty line followed
this is line 4,this is Third line
this is line 5,this is Fifth line
#使用a在第二行后插入文本
[root@localhost ~]# sed '2 a Insert' Sed.txt
this is line 1,this is First line
this is line 2,the Second line,Empty line followed
Insert
this is line 4,this is Third line
this is line 5,this is Fifth line
匹配行插入
#在匹配行的上一行插入问题
[root@localhost ~]# sed '/Second/i\Insert' Sed.txt
this is line 1,this is First line
Insert
this is line 2,the Second line,Empty line followed
this is line 4,this is Third line
this is line 5,this is Fifth line
6. 按需打印:p
使用p命令可进行打印,这里使用sed命令时一定要加-n参数,表示不打印没关系的行。
由于sed的工作原理是基于行的,因此每次都有大量的输出。可是这些输出中有一些是我们并不需要看到的,而只需要输出匹配的行或者处理过的行就好了。
sed的问题
#将the替换成THE
#sed实际处理了第二行,其他几行由于没有匹配所以并未真正处理
#但是sed的工作原理是基于流的,所以所有流过的行都打印出来了
[root@localhost ~]# sed 's/the/THE/' Sed.txt
this is line 1,this is First line
this is line 2,THE Second line,Empty line followed
this is line 4,this is Third line
this is line 5,this is Fifth line
打印指定的行
#打印出文件中指定的行
[root@localhost ~]# sed -n '1p' Sed.txt
this is line 1,this is First line
#使用p命令,则只打印实际处理过的行,简化了输出(使用-n参数)
[root@localhost ~]# sed -n 's/the/THE/p' Sed.txt
this is line 2,THE Second line,Empty line followed
三. sed脚本
定期对一些文件做分析操作,这种例行的工作往往有一定“标准化”的操作,比如说先去除文件中所有的空行,然后再全部替换某些字符等,这种过程类似于生产线上程式化的流水作业。
#该sed脚本的作用是将全文的this改为THAT,并删除所有空行
[root@localhost ~]# cat Sed.rules
s/this/THAT/g
/^$/d
[root@localhost ~]# sed -f Sed.rules Sed.txt
THAT is line 1,THAT is First line
THAT is line 2,the Second line,Empty line followed
THAT is line 4,THAT is Third line
THAT is line 5,THAT is Fifth line
四. 使用变量进行匹配替换
export new=good
echo "hello old frank" |sed "s/old/${new}/g"