linux日志分析

1.根据访问IP统计UV
awk -F ‘,’{print $1}’ house_access.log|sort | uniq -c |wc -l
命令解释: -F后边的’,'是指定输入分隔符(以日志中的逗号分隔变量)
日志内容如下:

11.11.11.11,60.1.147.128,-,13/Mar/2020:17:43:58 +0800,www.baidu.com,GET /Apinews/ArticleSlideAjax?_cb=jQuery1720045184271642938256_1584092647118&siteId=18692&_=1584092648041 HTTP/1.0,200,https://www.baidu.com/news/2020_03_13-52635033_0.shtml?from=news_concern_uleft,-,0.050,Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1

2.统计访问URL统计PV
awk -F"," ‘{print $6}’ house_access.log|wc -l

3.查询访问最频繁的URL

awk -F"," ‘{print $6}’ house_access.log|sort | uniq -c |sort -n -k 1 -r|more
4.查询访问最频繁的IP

awk -F ‘,’ ‘{print $2}’ house_access.log|sort | uniq -c |sort -n -k 1 -r|more

5.根据时间段统计查看日志

cat access.log| sed -n ‘/14/Mar/2015:21/,/14/Mar/2015:22/p’|more

日志内容如下:

[20200313 00:00:00] [java_api] [application] [1d979227265dd7f0] msg[Transfer Java Api Detail] urlkey[url地址] api[array('url'=>'url地址','dictId'=>278625,)] data[array('cost'=>0.023533999999999999,'http_code'=>200,'namelookup_time'=>0.0041599999999999996,'connect_time'=>0.007221,'pretransfer_time'=>0.019292,'starttransfer_time'=>0.023507,'redirect_time'=>0,'content'=>'{"data":{"address":"朝阳区管庄乡塔营村，项目四至：东至塔营中路，西至塔营西路，南至小寺村南街，北至塔营街。","alias":"","areaId":42,"areaName":"八里庄","bigCode":"4008185005","cloudBuyFlag":0,"dictName":"城志畅悦园","esfShowType":1,"id":278625,"lat":"39.903631","locationId":14,"locationName":"朝阳","lon":"116.605214","loupanType":1,"offsiteList":[],"price":29000.0,"priceUnit":"元/㎡","propertyTypeList":["保障房"],"salePhone":"01085364799","salePhoneAltPc":"32098","salePhoneAltWap":"94882","saleStatus":"售罄","showType":1,"siteId":3066,"siteName":"北京","state":1,"updateTime":1547554306},"errno":0,"msg":"操作成功"}',)] file[FrontApiService:169] _logtime[1584028800.9627] _env[fpm-fcgi] _ip[10.0.30.72] _uri[url地址?id=278625] _ua[mozilla/5.0 (windows nt 6.1) applewebkit/537.36 (khtml, like gecko) chrome/38.0.2125.104 safari/537.36]

6.匹配查找访问了最高的url 并输出对应的url

grep ‘url_地址*’ /data/logs/house/pc_java_api.0313.log | head | awk ‘{print $10}’
如果需要输出文件 命令后边加上 >>文件地址/文件名