文本处理之grep

grep 命令主要对文本的（正则表达式）行基于模式进行过滤

作用：文本搜索工具，根据用户指定的“模式”对目标文本逐行进行匹配检查；打印匹配到的行

模式：由正则表达式字符及文本字符所编写的过滤条件

格式：

grep [OPTIONS] PATTERN [FILE...]

常见选项：

选项	说明
-color=auto	对匹配到的文本着色显示
-m	匹配#次后停止
-v	显示不被pattern匹配到的行,即取反
-i	忽略字符大小写
-n	显示匹配的行号
-c	统计匹配的行数
-o	仅显示匹配到的字符串
-q	静默模式，不输出任何信息
-A #	after, 后#行
-B #	before, 前#行
-C #	context, 前后各#行
-e	实现多个选项间的逻辑or关系,如：grep –e ‘cat ’ -e ‘dog’ file
-w	匹配整个单词
-E	使用ERE，相当于egrep
-F	不支持正则表达式，相当于fgrep
-f file	根据模式文件处理
-r	递归目录，但不处理软链接
-R	递归目录，但处理软链接

范例：

grep root /etc/passwd

grep "USER" /etc/passwd

grep 'USER' /etc/passwd

grep whoami /etc/passwd

范例：取两个文件的相同行

[07:06:15 root@sz-centos7 /home]# cat f1.txt 
a
b
1
c
[07:06:19 root@sz-centos7 /home]# cat f2.txt 
b
e
f
c
1
2
[07:06:24 root@sz-centos7 /home]# grep -f /home/f1.txt /home/f2.txt 
b
c
1

范例: 分区利用率最大的值

[07:07:25 root@sz-centos7 /home]# df | grep '^/dev/sd' | tr -s ' ' % | cut -d% -f5 | sort -n | tail -n1

[07:10:40 root@sz-centos7 /home]# df | grep '^/dev/sd' | grep -oE '\<[0-9]{,3}%' | tr -d '%' | sort -nr | head -n1

[07:10:43 root@sz-centos7 /home]# df | grep '^/dev/sd' | grep -Eo '\<[0-9]{,3}%' | grep -Eo '[0-9]+' |sort -nr | head -n1

范例: 哪个IP和当前主机连接数最多的前三位

[root@centos8 ~]#ss -nt | grep "^ESTAB" |tr -s ' ' : |cut -d: -f6|sort |uniq -
c|sort -nr|head -n3

3 172.31.0.1
1 172.16.4.100
1 172.16.31.188

范例: 连接状态的统计

[07:14:23 root@sz-centos7 /home]# ss -tna | grep -v '^State' | cut -d" " -f1 | sort | uniq -c | sort -nr | head -n3
     35 LISTEN
     25 ESTAB
      4 TIME-WAIT
      
[07:15:23 root@sz-centos7 /home]# ss -tna | tail -n +2 | cut -d" " -f1 | sort | uniq -c
     25 ESTAB
     35 LISTEN
      4 TIME-WAIT

范例:

[07:14:23 root@sz-centos7 /home]# grep -v "^#" /etc/profile | grep -v '^$'

[07:14:23 root@sz-centos7 /home]# grep -v "^#\|^$" /etc/profile

[07:14:23 root@sz-centos7 /home]# grep -v "^\(#\|$\)" /etc/profile

[07:14:23 root@sz-centos7 /home]# grep -Ev "^(#|$)" /etc/profile

[07:14:23 root@sz-centos7 /home]# egrep -v "^(#|$)" /etc/profile

[07:14:23 root@sz-centos7 /home]# egrep -v '^(#|$)' /etc/httpd/conf/httpd.conf

范例:

[07:16:30 root@sz-centos7 /home]# grep -o 'r..t' /etc/passwd
root
root
root
root
r/ft

范例:

[07:20:30 root@sz-centos7 /home]# ifconfig eth0 | grep -E '[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}'
        inet 172.31.0.17  netmask 255.255.0.0  broadcast 172.31.255.255
        inet6 fe80::5f11:101c:653f:8a86  prefixlen 64  scopeid 0x20<link>
        RX packets 21149  bytes 17115402 (16.3 MiB)
        
[07:22:01 root@sz-centos7 /home]# ifconfig eth0 | grep -E '([0-9]{1,3}.){3}[0-9]{1,3}'
        inet 172.31.0.17  netmask 255.255.0.0  broadcast 172.31.255.255
        inet6 fe80::5f11:101c:653f:8a86  prefixlen 64  scopeid 0x20<link>
        RX packets 21377  bytes 17135654 (16.3 MiB)
        
[07:23:22 root@sz-centos7 /home]# ifconfig eth0 | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' | head -n1
172.31.0.17

范例:

[07:23:33 root@sz-centos7 /home]# grep -E 'root|bash' /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
hadoop:x:8000:8000::/home/hadoop:/bin/bash
cobbuser:x:8001:8001::/home/cobbuser:/bin/bash
webuser:x:8002:8002::/home/webuser:/bin/bash
long:x:8003:8003::/home/long:/bin/bash
longxuan:x:8004:8004::/home/longxuan:/bin/bash

范例：

[07:24:45 root@sz-centos7 /home]# grep -e 'root' -e 'bash' /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
hadoop:x:8000:8000::/home/hadoop:/bin/bash
cobbuser:x:8001:8001::/home/cobbuser:/bin/bash
webuser:x:8002:8002::/home/webuser:/bin/bash
long:x:8003:8003::/home/long:/bin/bash
longxuan:x:8004:8004::/home/longxuan:/bin/bash

范例：

[07:25:20 root@sz-centos7 /home]# grep -w root /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin

范例：

[07:26:03 root@sz-centos7 /home]# grep '\<root\>' /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin

范例：查找出头跟尾相同

[07:26:31 root@sz-centos7 /home]# grep "^\(.*\)\>.*\<\1$" /etc/passwd
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt

[07:27:45 root@sz-centos7 /home]# grep -E "^(.*)\>.*\<\1$" /etc/passwd
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt

[07:30:03 root@sz-centos7 /home]# egrep "^(.*)\>.*\<\1$" /etc/passwd
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt

范例：通过文本找出所有 ifconfig命令下的ip数字

[07:34:07 root@sz-centos7 /home]# cat ip_regex.txt 
([0-9]{1,3}\.){3}[0-9]{1,3}
[07:34:13 root@sz-centos7 /home]# ifconfig | grep -oEf ip_regex.txt 
172.17.0.1
255.255.0.0
172.17.255.255
172.31.0.17
255.255.0.0
172.31.255.255
127.0.0.1
255.0.0.0