服务器日志过滤uin-CSDN博客

问题

在服务器打印的log中有很多用户的uin，这些uin是命中了CGI的频率限制打印出来的，现在需要统计uin出现的次数进而判断它是否为异常uin。打印次数越多的uin异常可能性也就越大。

日志如下

<2> <(17445,17445,237)> 12:34:01 753 368694075 boardcast_err: Handle:158 check o2o security, to many times,  uin:368694075!
<2> (17445,17445,236)> 12:34:33 973 1434080321 boardcast_err: Handle:158 check o2o security, to many times,  uin:1434080321!
<2> <(17555,17555,154)> 12:34:44 565 368694075 boardcast_err: Handle:158 check o2o security, to many times,  uin:368694075!
<2> 19298,19298,74)> 12:34:56 130 368694075 boardcast_err: Handle:158 check o2o security, to many times,  uin:368694075!
<2> (17555,17555,149)> 12:35:26 863 368694075 boardcast_err: Handle:158 check o2o security, to many times,  uin:368694075!
<2> (19298,19298,79)> 12:35:49 607 368694075 boardcast_err: Handle:158 check o2o security, to many times,  uin:368694075!
<2> (17445,17445,239)> 12:36:43 643 368694075 boardcast_err: Handle:158 check o2o security, to many times,  uin:368694075!
<2> t(17445,17445,234)> 12:36:45 376 368694075 boardcast_err: Handle:158 check o2o security, to many times,  uin:368694075!
<2> <(19298,19298,78)> 12:37:18 659 368694075 boardcast_err: Handle:158 check o2o security, to many times,  uin:368694075!
<2> <(26627,26627,80)> 12:38:14 451 667024260 boardcast_err: Handle:158 check o2o security, to many times,  uin:667024260!
<2> <(24789,24789,236)> 12:39:48 993 667024260 boardcast_err: Handle:158 check o2o security, to many times,  uin:667024260!
<2> <(24789,24789,237)> 12:39:49 443 1461318781 boardcast_err: Handle:158 check o2o security, to many times,  uin:1461318781!
<2> <(24789,24789,223)> 12:41:52 133 667024260 boardcast_err: Handle:158 check o2o security, to many times,  uin:667024260!
<2> <(26627,26627,74)> 12:42:13 577 667024260 boardcast_err: Handle:158 check o2o security, to many times,  uin:667024260!
<2> <(24789,24789,237)> 12:42:14 365 667024260 ibeaconboardcast_err: Handle:158 check o2o security, to many times,  uin:667024260!

##过滤方法

用shell命令过滤和排序

 grep -o 'to many times,  uin:\([0-9]\+\)' 2015091012.log | sort | uniq -c

grep -o show only the part of a line matching PATTERN 完全匹配，只打印部分字段

uniq -c prefix lines by the number of occurrences 统计次数

sort 排序

      8 to many times,  uin:34080321
      1 to many times,  uin:61318781
   4842 to many times,  uin:12345678
      7 to many times,  uin:212405
      8 to many times,  uin:8694075
      5 to many times,  uin:7024260

可以得出12345678这个uin异常可能性最大。

看来后台开发学好awk,sed等命令很重要，需要入手一本《linux shell脚本攻略》。

##参考

Linux多线程服务端编程. Page. 111

https://m.oschina.net/blog/355303

转载于:https://my.oschina.net/lvyi/blog/504480