Anyone who has implemented a VPN has probably had to deal with MTU issues (unless you use a managed service). We’ve had code in our products for years to handle various MTU-related cases, and I’m going through some of it now. In double-checking our implementation, I took a peek at the relevant RFC, RFC 1191. In the introduction can be found this wonderful prediction:
It is expected that future routing protocols will be able to provide accurate PMTU information within a routing area, although perhaps not across multi-level routing hierarchies. It is not clear how soon that will be ubiquitously available, so for the next several years the Internet needs a simple mechanism that discovers PMTUs without wasting resources and that works before all hosts and routers are modified.
Woulda been nice…
At any rate, this is one of those IP features that is often handled incorrectly by firewalls. If you don’t allow Datagram Too Big messages (ICMP Type 3, Code 4) through, you are effectively killing performance (or even connectivity) for any paths with a Path MTU of less than your NIC’s MTU.
This is becoming more important: tunnel interfaces are proliferating, and as security consciousness increases, we’re only going to see more of this. For non-TCP-based tunneling protocols (IPSec, PPTP, GRE, L2TP, L2F, …) the carried packets are interface MTU minus overhead, so if you add, say, 100 bytes of overhead per packet, you wind up with a tunnel interface MTU of only 1400 bytes. Unless you want to have tons of frags, you have to listen for and handle Path MTU messages.
There are two common symptoms to path MTU problems:
- Connection speeds are bad and/or highly variable, caused by the fact that routers have to bounce your packets off of the line card and up to the CPU to fragment them (much slower), and by the fact that you’re doubling the number of packets to send. This also introduces the possibility of out-of-order delivery and the consequent fragment queuing. And to top everything off, fragmentation paths are never as well tested as non-frag paths, so there may be OS-level perf or security bugs you’re going to tickle.
- Connections suddenly quit working. While there are sometimes other explanations for this, a big cause of this problem is dropped datagrams due to MTU. TCP (usually) sets the DF bit on outgoing segments. If a router receives a segment that’s too big for the egress interface, and that segment has the DF bit set, it has no choice but to drop it. It then sends back its PMTU ICMP message, but the firewall drops that message, so the sender simply thinks that that the receiver has dropped off the face of the earth. Retransmissions happen and eventually the sender gives up. This generally happens with the first
big
packets in a session, such as (for example) when a terminal services client is painting the desktop.
So, if you’re an implementor of a firewall (or similar), pay attention to proper handling of Path MTU messages. And if you’re an end user and notice any of these symptoms, try removing firewalls and NAT devices and see if the problem disappears. It could be due to dropped PMTUD messages.