Re: PMTU-D: remember, your load balancer is broken

[Valdis.Kletnieks () vt edu]

On Tue, 13 Jun 2000 17:04:19 MDT, Marc Slemko <marcs () znep com>  said:
> Chances are that if you are using a load balancer for TCP connections,
> then it does not properly handle Path MTU Discovery.  Examples of devices

Does anybody have any field experience on how much PMTU-D actually
helps?  I just checked 'netstat -s' on an AIX box that runs a
stratum-2 NTP server, which accidentally had it enabled for
several weeks.  Abridged output follows:

ip:
        16357209 total packets received
        18411 fragments received
        5314999 path MTU discovery packets sent
        0 path MTU discovery decreases detected
icmp:
        Input histogram:
                echo reply: 3635421
                destination unreachable: 271455

AIX sends a test ICMP Echo to detect PMTU for UDP (which is where the
high icmp numbers came from).  The main interface on the box is a
10BaseT, so the MTU gets nailed to 1500.  As a result, I do *not* have
figures on how often we would have used a bigger MTU than 1500 - only
on whether there's still sub-1500 links out there.  On the other
hand, at least in today's Internet, the Other End is still quite
likely to be 10BaseT or PPP.

Approximately 80% of the traffic this machine sees is from off-campus,
all over the US.  We only got about 60% replies on the test ICMP Echo,
which constituted a good 40% of the entire traffic. In spite of this,
not once did the PMTU get fragmented below 1500.

Admittedly, PMTU-D for TCP is a lot less resource intensive (just
set the DF bit and see who salutes).  However, it should be tripped
roughly the same percent of the time (if a packet needs fragmenting,
it needs fragmenting - it's probably rare that a TCP packet of a given
size would fit but the same size UDP would fragment).

It looks to me like a better Rule Of Thumb is just:

a) If you know that the path to a specific net has an MTU over
1500 all the way, set a route specifying the MTU.

b) If you're a webserver or something else providing service Out
There to random users, just nail the MTU at 1500, which will
work for any Ethernet/PPP/SLIP out there.  And if you're load
balancing to geographically disparate servers, then your users
are probably Out There, with an MTU almost guaranteed to be 1500.

I assert that the chances of PMTU-D helping are in direct ratio to the
number of end users who have connections with MTU>1500 - it's almost
a sure thing that you probably won't have users with an MTU on their
last-hop that's bigger than their campus backbone and/or Internet
connection's MTU.

Is anybody seeing any documentable wins by using PMTU-D?

                                Valdis Kletnieks
                                Operating Systems Analyst
                                Virginia Tech