shell脚本语言编程进阶实战

最新推荐文章于 2019-05-22 16:52:40 发布

weixin_33923148

最新推荐文章于 2019-05-22 16:52:40 发布

阅读量121

点赞数

文章标签： shell awk 操作系统

原文链接：http://blog.51cto.com/19701522/1975062

版权

古人云“工欲善其事，必先利其器”。可以说，shell脚本语言编程就是系统管理员的一个编程利器、一个好帮手。熟练掌握和运用shell脚本语言编程，对于一个合格系统管理员来讲，可以说是基本素养。

shell脚本语言，可实现复杂程序逻辑，数据处理。除去强大的编程功能外，其无需编译环境的解释运行的特色，更显示其强大的灵活性。

学习shell编程，好处很多，但是困难是什么呢？有的说，没有shell编程学习范例。其实，在linux操作系统内，存在大量的脚本学习范例可供学习借鉴。

有的讲，没有编程需求。仔细回想一下，很多工作包含大量复杂的操作，是否有必要通过逻辑条件判断和数据抽取处理实现，并封装成自动化的脚本，以供今后重复调用。

在掌握shell编程的基础指令、正则表达和逻辑判断后，通过一些例子的练习，实现用户需求后，就可以达到shell脚本语言编程能力的进一步提升。.

本文向给出以下思路和框架。

实战.格式化文本数据

一些指令的多行输出结果，或一些文件的多行内容，可能已经结构化好了，但大部分没有格式化。所以编程进阶面对的一个重要问题，是抽取有用的数据并格式化数据，以便后续的加工处理。

格式化文本类数据基本方法如下：

（1）筛选数据

主要是运用grep、sed，抽取有用内容、剔除无用内容

# df -k|sed 1,2d 剔除指令输出中的前两行

# cat /etc/host| grep -v ^# |grep -v ^$ 剔除文件中的空行和注释行

# df -k |grep '[0-9][0-9]%' 抽取指令输出中带%且为2位数的行

# awk -F ":" '{print $1}' /etc/passwd 抽取文件的第一列

# df -k |grep '[0-9][0-9]%' |sed -e 's/%//g' 剔除指令输出中中%字符

# sed -i 's/APPLICATION//g' /etc/hosts 剔除文件中APPLICATION字符

（2）结构化数据

主要利用awk的指定字段分割符方法，完成数据的结构化

# ip a |grep "<" |awk -F ":" '{print $2}' |grep eth

（3）排序数据

主要是运用sort，对于文件差异的比对非常有用

# awk -F ":" '{print $1}' /etc/passwd |sort 按正序排列用户

# awk -F ":" '{print $1,$3}' /etc/group|sort -rn -k 2 按倒序排列用户组ID

实战.LINUX系统CPU空闲率指标检查

例：sar指令连续执行6次，求字段9%idle的最小值，如果小于80，则提示异常，否则显示正常：

脚本实现了外部参数的代入，通过统计后，得出判断结果

# cat check.sh

CPU_IDLE_STR="检查CPU空闲IDLE,大于百分"

CPU_IDLE_VAL=80

sar 1 6 |sed 1,3d| awk -v var1=$CPU_IDLE_STR -v var2=$CPU_IDLE_VAL 'BEGIN { min=0} {if ($9 < min ) min=(min<$9)?min:$9} END {if (min < var2) {printf "\t[ %-40s %-02d]\t\t*异常*\t\t当前值[ %-02d ] \n",var1,var2,min } else {printf "\t[ %-40s %-02d ]\t\t正常\t\t当前值[ %-02d ] \n",var1,var2,min }}'

# sh check.sh

例：sar指令连续执行6次，求字段9%idle的平均值，如果小于80，则提示异常，否则显示正常：

脚本实现了外部参数的代入，通过统计后，得出判断结果

# cat check.sh

CPU_IDLE_STR="检查CPU空闲IDLE,大于百分"

CPU_IDLE_VAL=80

sar 1 6 |sed 1,3d| awk -v var1=$CPU_IDLE_STR -v var2=$CPU_IDLE_VAL 'BEGIN { sum=0;count=0 } {sum +=$9; count++;} END {if (sum/count < var2) {printf "\t[ %-40s %-02d]\t\t*异常*\t\t当前值[ %-02d ] \n",var1,var2,sum/count } else {printf "\t[ %-40s %-02d ]\t\t正常\t\t当前值[ %-02d ] \n",var1,var2,sum/count }}'

# sh check.sh

例：统计不同日期内的报错

# cat m1.txt

Server1023/Server1023.log,Oct 17, 2017 7:31:12 PM GMT+08:00,Error> <Server1023>

Server1023/Server1023.log,Oct 17, 2017 7:32:12 PM GMT+08:00,Error> <Server1023>

Server1023/Server1023.log,Oct 17, 2017 8:05:20 PM GMT+08:00,Server1023> <[STUCK]

Server1113/Server1113.log,Sep 25, 2017 9:40:32 AM GMT+08:00,Error> <Server1113>

Server1113/Server1113.log,Sep 25, 2017 9:41:32 AM GMT+08:00,Error> <Server1113>

Server1113/Server1113.log,Sep 25, 2017 9:58:08 AM GMT+08:00,Server1113> <[STUCK]

Server1113/Server1113.log,Sep 29, 2017 3:59:22 PM GMT+08:00,Error> <Server1113>

Server1113/Server1113.log,Sep 29, 2017 4:00:22 PM GMT+08:00,Error> <Server1113>

# awk -F "," 'BEGIN { max="Oct 17" ;count=0 } { if ($2 == max ) { ++count } else { printf " %s %d \n",max,count ;max=$2;count=1} } END { printf " %s %d \n",max,count }' m1.txt

Oct 17 3

Sep 25 3

Sep 29 2

例：统计不同错误代码出现的次数

# cat m1.txt

####<Apr 7, 2017 2:07:27 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491545247275> <BEA-310003> <Free memory in the server is 4,905,048 bytes. There is danger of OutOfMemoryError>

####<Apr 8, 2017 9:06:33 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491613593096> <BEA-310003> <Free memory in the server is 4,728,536 bytes. There is danger of OutOfMemoryError>

####<Apr 9, 2017 3:46:38 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491680798625> <BEA-310003> <Free memory in the server is 4,992,928 bytes. There is danger of OutOfMemoryError>

####<Apr 9, 2017 10:54:44 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491749684634> <BEA-310003> <Free memory in the server is 4,772,736 bytes. There is danger of OutOfMemoryError>

####<Apr 10, 2017 5:51:50 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491817910249> <BEA-310003> <Free memory in the server is 4,794,072 bytes. There is danger of OutOfMemoryError>

####<Apr 11, 2017 12:51:55 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491886315817> <BEA-310003> <Free memory in the server is 4,960,776 bytes. There is danger of OutOfMemoryError>

####<Apr 12, 2017 7:52:01 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491954721248> <BEA-310003> <Free memory in the server is 4,989,464 bytes. There is danger of OutOfMemoryError>

####<Apr 13, 2017 2:55:06 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492023306984> <BEA-310003> <Free memory in the server is 4,845,776 bytes. There is danger of OutOfMemoryError>

####<Apr 13, 2017 9:59:12 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492091952865> <BEA-310003> <Free memory in the server is 4,833,352 bytes. There is danger of OutOfMemoryError>

####<Apr 14, 2017 5:00:05 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492160405970> <BEA-310003> <Free memory in the server is 4,979,432 bytes. There is danger of OutOfMemoryError>

####<Apr 15, 2017 12:04:11 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492229051734> <BEA-310003> <Free memory in the server is 4,957,360 bytes. There is danger of OutOfMemoryError>

####<Apr 16, 2017 7:08:17 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492297697464> <BEA-310003> <Free memory in the server is 4,798,184 bytes. There is danger of OutOfMemoryError>

####<Sep 23, 2017 9:41:39 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1506174099718> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>

####<Sep 23, 2017 9:41:39 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1506174099719> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>

####<May 27, 2017 9:36:29 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495892189359> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>

####<May 27, 2017 9:36:29 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495892189360> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>

####<May 27, 2017 10:02:56 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495893776640> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>

####<May 27, 2017 10:02:56 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495893776641> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>

####<May 28, 2017 7:04:24 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495969464161> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>

####<May 28, 2017 7:04:24 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495969464161> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>

# awk -F"<" '/BEA/ {print $6,$13 } END {}' o1.txt |awk '{printf "\t%s %s %s\n",S[$NF],$1,$2;++S[$NF];printf "\t %s %s %s\n",S[$NF],$1,$2 } END {for(a in S) printf "END %s %d \n",a,S[a]}'

END BEA-000415> 4

END BEA-000416> 4

END BEA-310003> 12

转载于:https://blog.51cto.com/19701522/1975062