接着昨天的步子。今天写完监控和限流。主要通过解析tcp_access.log来实时监控各个端口的网络流量情况。对于流量超标的端口暂时封闭,隔天恢复。
- Nginx TCP日志字段解析
log_time worker_process_pid client_ip host_ip accept_time upstream_ip bytes_read bytes_write
2011/08/02 06:19:07 [5972] 127.0.0.1 0.0.0.0:1982 2011/08/02 06:18:19 172.19.0.129:80 80 236305
log_time: The current time when writing this log.
The log action is called when the proxy session is closed.
worker_process_pid: the pid of worker process
client_ip: the client ip
host_ip: the server ip and port
accept_time: the time when the server accepts client's connection
upstream_ip: the upstream server's ip
bytes_read: the bytes read from client
bytes_write: the bytes written to client
- Python日志解析及监控脚本
# -*- coding: utf-8 -*-
"""
"""
import os
import time
import json
INPUT=0.0
OUTPUT=0.0
TATAL=0.0
PORT_MAP = {}
FORBID_MAP = {}
LIMIT = 8192.0
f=open("/usr/local/nginx/logs/tcp_access.log","r")
l = open("/data/nginx/forbid.list", "a+")
for fine in l :
FORBID_MAP[fine.replace("\n","")]=fine.replace("\n","")
for line in f :
arr = line.split(" ")
PORT_MAP
port = arr[4].split(":")[1]
if PORT_MAP.has_key(port):
PORT_MAP[port]["INPUT"] += float(arr[8])
PORT_MAP[port]["OUTPUT"] += float(arr[9])
else :
PORT_MAP[port] = {
"INPUT" : float(arr[8]),
"OUTPUT" : float(arr[9])
}
INPUT += long(arr[8])
OUTPUT += long(arr[9])
TATAL += long(arr[8]) + long(arr[9])
for (k,v) in PORT_MAP.items():
v["LOGTIME"] = time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(time.time()))
v["TATAL"] = round((v["INPUT"]+v["OUTPUT"])/1024/1024,2)
v["INPUT"] = round(v["INPUT"]/1024/1024,2)
v["OUTPUT"] = round(v["OUTPUT"]/1024/1024,2)
if v["TATAL"] > LIMIT and (not FORBID_MAP.has_key(k)):
os.system("firewall-cmd --permanent --remove-port="+k+"/tcp ")
os.system("firewall-cmd --reload ")
l.write(k+"\n")
PORT_MAP["SUMMARY"] = {
"LOGTIME" : time.strftime('%Y-%m-%d %H:%M:%S',time.localtime(time.time())),
"INPUT" : round(INPUT/1024/1024,2),
"OUTPUT" : round(OUTPUT/1024/1024,2),
"TATAL" : round(TATAL/1024/1024,2)
}
JSONSTR = json.dumps(PORT_MAP);
w = open("/data/nginx/result.json", "a+")
w.write(JSONSTR+",\n")
脚本会将日志解析为端口的流量信息实时汇总。格式为:
{
"1234": {
"INPUT": 0.05,
"TATAL": 0.39,
"LOGTIME": "2016-03-22 02:52:07",
"OUTPUT": 0.34
},
"2234": {
"INPUT": 31.66,
"TATAL": 753.82,
"LOGTIME": "2016-03-22 02:52:07",
"OUTPUT": 722.16
},
"SUMMARY": {
"INPUT": 31.71,
"TATAL": 754.21,
"LOGTIME": "2016-03-22 02:52:07",
"OUTPUT": 722.5
}
}
配合一个5秒循环脚本,定时监控流量。如端口流量超标,通过防火墙关闭该端口。
#!/bin/bash
while true
do
python /usr/local/shell/analyst.py
sleep 5
done
- Nginx 日志切分和端口重开
#!/bin/bash
# 1. move host.access.log to host.access_20120821.log
logs_path="/usr/local/nginx/logs/"
mv ${logs_path}access.log ${logs_path}access_$(date -d "yesterday" +"%Y%m%d").log
mv ${logs_path}tcp_access.log ${logs_path}tcp_access_$(date -d "yesterday" +"%Y%m%d").log
kill -USR1 `cat ${logs_path}nginx.pid`
# 2. delete all host.access_2012xxxx.log which overtime 10 days
#find ${logs_path} -name 'access_*.log' -type f -mtime +10 -exec rm {} \;
# clear forbid.list ,reopen ports
> /data/nginx/forbid.list
firewall-cmd --permanent --add-port=1234/tcp
firewall-cmd --permanent --add-port=2234/tcp
firewall-cmd --permanent --add-port=3234/tcp
firewall-cmd --permanent --add-port=4234/tcp
firewall-cmd --reload
配置定时任务每天凌晨执行
1 0 * * * sh /usr/local/shell/nginxcutlogs.sh