IP MTU and TCP MSS Missmatch – an evil for network performance

When we conduct a technical workshop, a common query from the participants relates to the Maximum Transmission Unit (MTU) size manipulation on a router interface and its relationship with the TCP Maximum Segment Size (MSS). I will try to discuss this in detail from a network engineers point of view.

The Maximum Transmission Unit (MTU) is the maximum length of data that can be transmitted by a protocol in one instance. If we take the Ethernet interface as an example, the MTU size of an Ethernet interface is 1500 bytes by default, which excludes the Ethernet frame header and trailer. It means that the interface cannot carry any frame larger then 1500 bytes. If we look inside the frame, we have a 20 byte IP header + 20 byte TCP header, leaving a 1460 byte of the payload that can be transmitted in one frame. This is what we refer to as TCP MSS. The diagram below visualizes this concept:

 

 

 

In a normal transmission case, if there is no additional encapsulation (that is, IPsec, MPLS and so forth) carried out on a transiting router, the source device may use the maximum payload length of 1460 bytes without any potential risk of packet fragmentation/drop. This is negotiated during the TCP three-way handshake stage between the source and destination host.

Now in a likely case where a router in transit is carrying out additional encapsulation, that is, MPLS label swapping, this will add an additional label header as below:

 

 

 

This will eventually increase the size of the frame exiting a transiting router (in the case above it is 1508 bytes.) It will also create similar issue for IPSec and IPv6 in IPv4 tunnel etc.

The maximum MTU of an interface will depend on the hardware platform, but the IEEE 802.3 standards require a minimum MTU of 1500 bytes. Also if you notice the following CLI on a router, the maximum IP MTU is capped at the Ethernet MTU of 1500 bytes.

 

The problem is we cannot increase the IP MTU size on the router Ethernet interface because the MPLS label encapsulation frame size can potentially exceed the maximum MTU. If the source device creates a full size packet with a TCP MSS of 1460 bytes, it is likely this transiting router will drop/fragment the packet. This is bad for our network performance.

What is the best way to solve this?

Shrinking the IP MTU on the interface to 1448 bytes  will create space for 12 additional bytes on the MPLS label header [if you consider a maximum of 3 MPLS label LDP + VPN + TE] plus a 20 byte TCP + 20 byte IP header within a 1500 byte Ethernet MTU.

Please remember, when we shrink the IP MTU on a transiting router, that will also shrink the maximum possible TCP MSS size to 1448 bytes from the source device. If the router does not signal the source and destination during the TCP handshake, the optimal TCP MSS  could potentially create a dropping/fragmentation problem.

Here are the steps to solve this:

Transiting traffic (traffic going via this router):

Router1(config)#int ethernet 1/0
Router1(config-if)#ip mtu 1448

The above command will shrink the IP MTU on the interface to 1448 bytes, creating a 12 byte space for the MPLS label header.

Router1(config-if)#ip tcp adjust-mss ?
<500-1460>
Router1(config-if)#ip tcp adjust-mss 1448

The above command will signal the source and destination device during the three-way handshake to use the TCP MSS size of 1448 bytes so that if they create the full size packet there will still not be any drop/fragmentation on the router. 

Terminating traffic (doing SSH, telnet to this router):

Router1(config)# ip tcp mss 1448?
<68-10000>  MSS

I do hope this blog piece will help you to understand how to handle the IP MTU and TCP MSS size on a router interface.

 

Read another technical guide by Nurul.

出处：

http://blog.apnic.net/2014/12/15/ip-mtu-and-tcp-mss-missmatch-an-evil-for-network-performance/