1. Review current connections
- Check which port has the most of TW connections
- Dump all
time-wait
connections to a file
# ss -antp state time-wait > /tmp/tw-connections-`date +%Y%m%d`
- Get local TW for the local ports
# cat /tmp/tw-connections-`date +%Y%m%d` | awk '{print $3}' | sort | uniq -c | sort -k 1 -n -r | head
-
- Example output
2619 10.0.20.15:80
711 127.0.0.1:28080
496 127.0.0.1:8080
438 127.0.0.1:18080
12 10.0.20.15:10050
-
- Local Port 80 has 2619 connections on Time-Wait
- Get remote TW for the local ports
# cat /tmp/tw-connections-`date +%Y%m%d` | awk '{print $4}' | sort | uniq -c | sort -k 1 -n -r | head
- Check number of connections that are in TIME_WAIT state.
# wc -l /tmp/tw-connections-`date +%Y%m%d`
- Add to the escalation request
- Get a summary of current connections
# ss -s
2. Verify kernel setting for tcp_max_tw_buckets.
# cat /proc/sys/net/ipv4/tcp_max_tw_buckets
- The correct value is 180000
180000
3. Verify kernel setting for tcp_fin_timeout.
# cat /proc/sys/net/ipv4/tcp_fin_timeout
- By our standards it should be 30
30
4. If TCP related kernel setting are not properly set, we need to modify those setting manually.
- Check if appropriate kernel setting exist in sysctf.conf file
# cat /etc/sysctl.conf | grep tcp_max_tw_buckets
net.ipv4.tcp_max_tw_buckets = 5000
# cat /etc/sysctl.conf | grep net.ipv4.tcp_fin_timeout
net.ipv4.tcp_fin_timeout = 30
- Modify values in /etc/sysctl.conf file
# vi /etc/sysctl.conf
net.ipv4.tcp_max_tw_buckets = 180000
net.ipv4.tcp_fin_timeout = 30
- Apply changes and check current TW max bucket value
# sysctl -p
# cat /proc/sys/net/ipv4/tcp_max_tw_buckets
# cat /proc/sys/net/ipv4/tcp_fin_timeout
- Monitor logs and number of Time Wait connections for some time.