一次内核网络listendrops分析记录

最新推荐文章于 2023-04-28 11:49:55 发布

chenzhOS

最新推荐文章于 2023-04-28 11:49:55 发布

阅读量1.6k

点赞数

分类专栏：杂项

本文链接：https://blog.csdn.net/weixin_40776947/article/details/113629262

版权

杂项专栏收录该内容

5 篇文章 1 订阅

订阅专栏

背景如下：手上有一个机顶盒开发板，于是想通过adb连接进去进行各种操作。

1. 机顶盒开机，设置静态ip:192.168.10.99 pc的ip: 192.168.10.88

eth0 Link encap:Ethernet HWaddr
inet addr:192.168.10.99 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::232:88ff:fe00:1/64 Scope:Link
UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
RX packets:141 errors:0 dropped:0 overruns:0 frame:0
TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:18654 (18.2 KiB) TX bytes:9656 (9.4 KiB)
Interrupt:209 Base address:0x4000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

2. 网络测试：

:/ # ping 192.168.10.88

PING 192.168.10.88 (192.168.10.88) 56(84) bytes of data.
64 bytes from 192.168.10.88: icmp_seq=1 ttl=128 time=3.38 ms
64 bytes from 192.168.10.88: icmp_seq=2 ttl=128 time=3.72 ms
^C
--- 192.168.10.88 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 3.389/3.556/3.723/0.167 ms

3. 以为网络正常后，于是开心的启用adb连接，没想到发生了错误：

C:\Users>adb connect 192.168.10.99
unable to connect to 192.168.10.99:5555

4. 看到这个错误，于是想起看看机顶盒adb是否开启，监听端口是否正常：

:/ # ps | grep adb

shell 1180 1 9816 136 ffffffff 00026b98 S /sbin/adbd

:/ # netstat -lanp

Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:554 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:5555 0.0.0.0:* LISTEN

5. 机顶盒监听也正常，只要开启抓包大法了：

（不方便上图，就说下结果吧），看到pc发了一次syn和两次syn retransimission,也就是说PC发过去的报文，机顶盒没有回复syn+ack.

6. 头大了，这难道是内核tcp协议栈的问题？苦思冥想了很久。

7. 那就从机顶盒抓包分析吧：

[ 118.289264] device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
20:01:57.283309 IP 192.168.10.88.53761 > 192.168.10.99.rplay: Flags [S], seq 2250075644, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
20:02:00.284338 IP 192.168.10.88.53761 > 192.168.10.99.rplay: Flags [S], seq 2250075644, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
20:02:06.284411 IP 192.168.10.88.53761 > 192.168.10.99.rplay: Flags [S], seq 2250075644, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0

:/ # busybox ifconfig

eth0 Link encap:Ethernet HWaddr 00:32:88:00:00:01
inet addr:192.168.10.99 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::232:88ff:fe00:1/64 Scope:Link
UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
RX packets:59 errors:0 dropped:0 overruns:0 frame:0
TX packets:37 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8214 (8.0 KiB) TX bytes:9642 (9.4 KiB)
Interrupt:209 Base address:0x4000

8. 可见网卡接收正常，难道是防火墙？清空防火墙表，默认全部ACCEPT

:/ # iptables -F -t raw
:/ # iptables -F -t nat
:/ # iptables -F -t mangle
:/ # iptables -F -t filter

9. 问题依旧，这就头大了。因为是机顶盒，netstat -s命令不存在的，只好看下内核源码关于tcp报文统计，没想到还真有：

:/ # cat /proc/net/netstat

TcpExt: SyncookiesSent SyncookiesRecv SyncookiesFailed EmbryonicRsts PruneCalled RcvPruned OfoPruned OutOfWindowIcmps LockDroppedIcmps ArpFilter TW TWRecycled TWKilled PAWSPassive PAWSActive PAWSEstab DelayedACKs DelayedACKLocked DelayedACKLost ListenOverflows ListenDrops TCPPrequeued TCPDirectCopyFromBacklog TCPDirectCopyFromPrequeue TCPPrequeueDropped TCPHPHits TCPHPHitsToUser TCPPureAcks TCPHPAcks TCPRenoRecovery TCPSackRecovery TCPSACKReneging TCPFACKReorder TCPSACKReorder TCPRenoReorder TCPTSReorder TCPFullUndo TCPPartialUndo TCPDSACKUndo TCPLossUndo TCPLostRetransmit TCPRenoFailures TCPSackFailures TCPLossFailures TCPFastRetrans TCPForwardRetrans TCPSlowStartRetrans TCPTimeouts TCPLossProbes TCPLossProbeRecovery TCPRenoRecoveryFail TCPSackRecoveryFail TCPSchedulerFailed TCPRcvCollapsed TCPDSACKOldSent TCPDSACKOfoSent TCPDSACKRecv TCPDSACKOfoRecv TCPAbortOnData TCPAbortOnClose TCPAbortOnMemory TCPAbortOnTimeout TCPAbortOnLinger TCPAbortFailed TCPMemoryPressures TCPSACKDiscard TCPDSACKIgnoredOld TCPDSACKIgnoredNoUndo TCPSpuriousRTOs TCPMD5NotFound TCPMD5Unexpected TCPSackShifted TCPSackMerged TCPSackShiftFallback TCPBacklogDrop TCPMinTTLDrop TCPDeferAcceptDrop IPReversePathFilter TCPTimeWaitOverflow TCPReqQFullDoCookies TCPReqQFullDrop TCPRetransFail TCPRcvCoalesce TCPOFOQueue TCPOFODrop TCPOFOMerge TCPChallengeACK TCPSYNChallenge TCPFastOpenActive TCPFastOpenPassive TCPFastOpenPassiveFail TCPFastOpenListenOverflow TCPFastOpenCookieReqd TCPSpuriousRtxHostQueues
TcpExt: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
IpExt: InNoRoutes InTruncatedPkts InMcastPkts OutMcastPkts InBcastPkts OutBcastPkts InOctets OutOctets InMcastOctets OutMcastOctets InBcastOctets OutBcastOctets InCsumErrors
IpExt: 29 0 0 0 13 0 14215 4788 0 0 1385 0 0

看到了Listendrops居然统计6，这跟我发了执行了两次adb connect发了6个包对的上。但是为什么会listendrops呢，网上也没有很详细的答案。只好搜源码了。

:/mnt/e/linux-3.16.84/net$ grep "listendrops" -nri .
./dccp/ipv4.c:443:      NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);
./dccp/ipv6.c:620:      NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);
./ipv4/proc.c:207:      SNMP_MIB_ITEM("ListenDrops", LINUX_MIB_LISTENDROPS),
./ipv4/tcp_ipv4.c:493:          NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);
./ipv4/tcp_ipv4.c:1399: NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);
./ipv4/tcp_ipv4.c:1494: NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);
./ipv6/tcp_ipv6.c:442:          NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);
./ipv6/tcp_ipv6.c:1130: NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);
./ipv6/tcp_ipv6.c:1331: NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENDROPS);

10. 于是几乎可以确定是tcp_ipv4.c这个文件了，然后仔细分析了493和1399和1494这几行，发生的条件

if (sk_acceptq_is_full(sk)) {
		NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENOVERFLOWS);
		goto drop;  //这个是说accept满了，这不可能呀，我的cpu是几乎是空闲的，内存也没占用多少

if ((sysctl_tcp_syncookies == 2 ||
	     inet_csk_reqsk_queue_is_full(sk)) && !isn) {
		want_cookie = tcp_syn_flood_action(sk, skb, "TCP");
		if (!want_cookie)
			goto drop;
	}  //这也不太可能，syn_cookies我没编译进去

if (tmp_opt.saw_tstamp &&
		    tcp_death_row.sysctl_tw_recycle &&
		    (dst = inet_csk_route_req(sk, &fl4, req)) != NULL &&
		    fl4.daddr == saddr) {
			if (!tcp_peer_is_proven(req, dst, true)) {
				NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_PAWSPASSIVEREJECTED);
				goto drop_and_release;
			}
		} //这好像有点可能，尝试了timestamp关闭了也不管用，这里好像检查了route，难道是路由表的问题？

11. 于是怀疑了路由表的问题：

:/ # busybox route

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.10.0 * 255.255.255.0 U 0 0 0 eth0

12. 好像也没问题啊，难道是策略路由？

:/ # ip rule list

0: from all lookup local
10000: from all fwmark 0xc0000/0xd0000 lookup legacy_system
13000: from all fwmark 0x10063/0x1ffff lookup local_network
15000: from all fwmark 0x0/0x10000 lookup legacy_system
16000: from all fwmark 0x0/0x10000 lookup legacy_network
17000: from all fwmark 0x0/0x10000 lookup local_network
23000: from all fwmark 0x0/0xffff uidrange 0-0 lookup main
32000: from all unreachable

13. 还真是，赶紧添加上去试一下：

:/ # ip rule add from 192.168.10.0/24 table main

14. 问题解决：

C:\Users\>adb connect 192.168.10.99
connected to 192.168.10.99:5555

chenzhOS

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
一次内核网络listendrops分析记录

背景如下：手上有一个机顶盒开发板，于是想通过adb连接进去进行各种操作。1. 机顶盒开机，设置静态ip:192.168.10.99 pc的ip: 192.168.10.88eth0 Link encap:Ethernet HWaddr inet addr:192.168.10.99 Bcast:192.168.10.255 Mask:255.255.255.0 inet6 addr: fe80::232:88ff:fe00:1/64 Sc...
复制链接

扫一扫