实验验证为什么tcpdump输出看到cksum错误（by quqi99)

 

作者：张华  发表于：2016-04-27
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明

 

( http://blog.csdn.net/quqi99 )

 

 

 

问题

经常看到tcpdump显示校验错误（cksum 0x0a2b (incorrect -> xxx），如下：
hua@node1:~$ tcpdump -vvv -s0 host 172.30.97.21 
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 


# TCP three-way handshake 
16:52:26.924380 IP (tos 0x0, ttl 64, id 63899, offset 0, flags [DF], proto TCP (6), length 60) 
172.30.80.170.35828 > 172.30.97.21.https: Flags [S], cksum 0x0a2b (incorrect -> 0xde6a), seq 1478381982, win 29200, options [mss 1460,sackOK,TS val 236431365 ecr 0,nop,wscale 7], length 0 
16:52:26.925651 IP (tos 0x0, ttl 62, id 0, offset 0, flags [DF], proto TCP (6), length 60) 
172.30.97.21.https > 172.30.80.170.35828: Flags [S.], cksum 0x17cf (correct), seq 3506721850, ack 1478381983, win 27460, options [mss 1385,sackOK,TS val 1812551 ecr 236431365,nop,wscale 7], length 0 
16:52:26.925674 IP (tos 0x0, ttl 64, id 63900, offset 0, flags [DF], proto TCP (6), length 52) 
172.30.80.170.35828 > 172.30.97.21.https: Flags [.], cksum 0x0a23 (incorrect -> 0xb0af), seq 1, ack 1, win 229, options [nop,nop,TS val 236431365 ecr 1812551], length 0 


# .170 sends data (length=273) to .21, then .21 confirms with ACK=SEQ+LENGTH=274 
16:52:26.926089 IP (tos 0x0, ttl 64, id 63901, offset 0, flags [DF], proto TCP (6), length 325) 
172.30.80.170.35828 > 172.30.97.21.https: Flags [P.], cksum 0x0b34 (incorrect -> 0x1aec), seq 1:274, ack 1, win 229, options [nop,nop,TS val 236431365 ecr 1812551], length 273 
16:52:26.926458 IP (tos 0x0, ttl 62, id 64636, offset 0, flags [DF], proto TCP (6), length 52) 
172.30.97.21.https > 172.30.80.170.35828: Flags [.], cksum 0xafa4 (correct), seq 1, ack 274, win 223, options [nop,nop,TS val 1812551 ecr 236431365], length 0 


# .21 sends data (length=1151) to .170, then .170 confirms with ACK=SEQ+LENGTH=1152 
16:52:26.931754 IP (tos 0x0, ttl 62, id 64637, offset 0, flags [DF], proto TCP (6), length 1203) 
172.30.97.21.https > 172.30.80.170.35828: Flags [P.], cksum 0xed16 (correct), seq 1:1152, ack 274, win 223, options [nop,nop,TS val 1812552 ecr 236431365], length 1151 
16:52:26.931777 IP (tos 0x0, ttl 64, id 63902, offset 0, flags [DF], proto TCP (6), length 52) 
172.30.80.170.35828 > 172.30.97.21.https: Flags [.], cksum 0x0a23 (incorrect -> 0xab07), seq 274, ack 1152, win 251, options [nop,nop,TS val 236431366 ecr 1812552], length 0 
 

实验

hua@node1:~$ sudo ethtool --offload eth0 rx on tx on sg on tso on
hua@node1:~$ sudo ethtool -k eth0 |grep on
rx-checksumming: on
tx-checksumming: on
tx-checksum-ip-generic: on
scatter-gather: on
tx-scatter-gather: on阶段
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
rx-vlan-offload: on
tx-vlan-offload: on
receive-hashing: on
highdma: on [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]


hua@node1:~$ nslookup twitter.com


hua@node1:~$ sudo tcpdump -i eth0 -vvv -nn udp dst port 53
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
18:06:33.189154 IP (tos 0x0, ttl 64, id 5717, offset 0, flags [none], proto UDP (17), length 57)
    192.168.99.124.41654 > 192.168.99.1.53: [bad udp cksum 0x4805 -> 0x1e86!] 10190+ A? twitter.com. (29)


hua@node1:~$ sudo ethtool --offload eth0 rx off tx off sg off tso off
Actual changes:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ip-generic: off
scatter-gather: off
tx-scatter-gather: off
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp6-segmentation: off
generic-segmentation-offload: off [requested on]
hua@node1:~$ sudo ethtool -k eth0 |grep off
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: off
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]


hua@node1:~$ sudo tcpdump -i eth0 -vvv -nn udp dst port 53
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
18:07:31.536313 IP (tos 0x0, ttl 64, id 10150, offset 0, flags [none], proto UDP (17), length 57)
    192.168.99.124.42730 > 192.168.99.1.53: [udp sum ok] 61434+ A? twitter.com. (29)

附件，如何解读tcpdump输出

hua@node1:~$ tcpdump -n port 8000
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

//内容格式
//时分秒           客户端IP和端口         服务端IP和端口   SYN(客户端主动请求连接服务器)  包序号  8192(Window size),  长度0
11:12:02.534303 IP 192.168.1.109.8886 > 192.168.1.178.irdmi: Flags [S], seq 2639909369, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0

//服务器端发送SYN+ACK(.)的报文给客户端,  ack=seq+1=2639909369+1,  长度0
11:12:02.534339 IP 192.168.1.178.irdmi > 192.168.1.109.8886: Flags [S.], seq 1984808639, ack 2639909370, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0

//客户端收到确认包后，发送ACK(.)报文给服务端,  长度0
11:12:02.534528 IP 192.168.1.109.8886 > 192.168.1.178.irdmi: Flags [.], ack 1, win 256, length 0

//三次握手成功后,  客户端发送"11111"报文内容给服务端, 长度为5字节, 标识P即PSH(Push function)为1时, 代表要发送缓冲区的封包, 无需等待缓冲区满了才送
11:12:16.501177 IP 192.168.1.109.8886 > 192.168.1.178.irdmi: Flags [P.], seq 1:6, ack 1, win 256, length 5

//服务端收到客户端请求报文包后, 发送ACK(.)报文给客户端,  长度为0字节,  ACK=SEQ+LENGTH=1+5
11:12:16.501208 IP 192.168.1.178.irdmi > 192.168.1.109.8886: Flags [.], ack 6, win 115, length 0

//客户端继续发送请求报文, 内容为"123321", 长度6字节
11:44:16.181474 IP 192.168.1.109.8886 > 192.168.1.178.irdmi: Flags [P.], seq 6:12, ack 1, win 256, length 6

//服务端发送ACK(.)应答报文给客户端, ack=SEQ+length=6+6=12
11:44:16.181508 IP 192.168.1.178.irdmi > 192.168.1.109.8886: Flags [.], ack 12, win 115, length 0

//客户端关闭后向服务端发送标识为(R+.ACK), RST(Reset)为1时, 表示tcp连接马上会被结束，而无需等待终止确认手续. 也就是说, 这是个强制结束的连线, 且发送端已经断开.
//这里即使服务端没有调用close(accept_sockfd)来向客户端发送FIN报文，两者之间建立的tcp链路都会被关闭！！！
11:50:55.047977 IP 192.168.1.109.8886 > 192.168.1.178.irdmi: Flags [R.], seq 12, ack 1, win 0, length 0

附件 - ovs-tcpdump

sudo ovs-tcpdump -i br-int "(arp or icmp) and net 192.168.100./24 and ether host fa:16:3e:51:a6:8a" -p ovs_tcpdump_capture.pcap 

 tshark -r rmq-machine10-tcpdump-SF250329.pcap |awk '{arr[$5]++}END{for (a in arr) print a, arr[a]}' |sort -n -k 2 -r | head -n 3

附件 - 抓vm的tap流量

tcpdump -l -i tap1537bd14-c2 "((arp or icmp) or (udp and port 53))" -w `hostname`_tap1537bd14-c2_arp_icmp_dns.pcap
sudo ip n flush all
mtr -n --tcp -P 80 --report 91.189.88.180
sudo ovs-appctl ofproto/trace br-int in_port=188,arp,arp_spa=10.10.100.190,dl_src=fa:16:3e:91:5f:c3 > `hostname`_ofproto_trace_brint_arp.out

tshark -2 -cdk09* -R "(eth.addr == fa:16:3e:4d:11:86 or eth.addr == fa:16:3e:88:d6:2f) and icmp"