一、nginx的日志格式
查看nginx.conf 文件,默认格式如下:
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
字段说明
下面具体看下访问日志的内容:
223.104.41.37 - - [05/Jul/2022:13:34:20 +0800] "GET /api/book/info?bookId=123 HTTP/1.1" 200 14632 "http://www.zzz.com.cn/archive?bookId=123" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36"
解释:
远程主机IP地址 访问时间 时区 方法 资源 协议 状态码 发送字节 Referer 浏览器信息
二、access.log 文件位置
nginx.conf 中内容:
access_log /var/log/nginx/access.log main;
说明我们的日志位置在/var/log/nginx 下。
三、日志分析:
1、统计前5的访问IP
# awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -5
7093 183.152.124.55
3719 218.108.36.18
1797 115.220.140.234
1545 112.10.236.137
1141 183.228.110.80
2、统计指定某一天的访问IP
# grep "18/May/2022" /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -nr | head -5
755 112.10.236.127
358 223.94.216.200
348 116.30.149.23
283 140.243.118.204
270 183.253.242.192
# awk '/18\/May\/2022/ {print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -5
755 112.10.236.127
358 223.94.216.200
348 116.30.149.23
283 140.243.118.204
270 183.253.242.192
文件较大的时候,建议先grep再awk,这样速度快很多。
3、统计指定资源
处理第7个字段以'.html'结尾的行
# awk '$7 ~ /\.html$/ {print $1,$7,$9}' /var/log/nginx/access.log
14.104.225.143 /web/common/success.html 200
219.153.191.189 /web/common/success.html 200
152.32.189.96 /mtja.html 200
152.32.189.96 /index.html 200
152.32.189.96 /login.html 200
152.32.189.96 /mindex.html 200
4、过滤URL
$ awk '{print $11}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -5
12133 "http://www.zzz.com.cn/translation"
7550 "http://www.zzz.com.cn/applicationAdd"
4255 "http://www.zzz.com.cn/search"
2565 "http://www.zzz.com.cn/request"
2257 "http://www.zzz.com.cn/order"
5、统计流量
$ grep "03/Jul/2022" /var/log/nginx/access.log | awk '{sum+=$10} END{print sum}'
54827188
6、统计状态码
$ awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -10
77065 200
2933 304
1519 400
148 405
106 206
65 499
9 173
5 408
2 504
2 404