Re: MTU issue routing traffic via Cisco GRE tunnel to Nokia/Check Point firewall

[Eric Vyncke <evyncke () cisco com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Late reply... sorry about it.

It looks like you are experimenting a Path MTU Discovery issue. As it is quite possible that firewalls (or even a load 
balancers somewhere -- like in Yahoo) simply drop the required ICMP messages.

The simple solutions are:
- - use 'ip tcp adjust-mss 1400' on a router seeing traffic in the clear to force MSS to 1400 so IP datagram size to 
1420 (of course 1400 is just a guess), this will cover all TCP traffic
- - set 'ip mtu 1500' on the GRE tunnel interface (yes 1500 bytes)

The latter will also work for non TCP traffic. So, you probably need both.

Of course, the 'clean' solution is to let the ICMP unreachable too big messages go through.

Reasoning:
- - GRE encapsulation clears the DF bit UNLESS 'tunnel path-mtu-discovery' is set on the tunnel interface (if turned on 
the tunnel MTU will be dynamically adjusted upon receipt of ICMP)
- - IPsec encapsulation copies the DF and adjusts the path MTU upon receipt of ICMP UNLESS 'crypto ipsec df-bit 
clear/set' is configured in the crypto map
- - router will fragment when forwarding to any interface whose MTU is smaller than the received IP packet. This 
happens often when forwarding to a GRE tunnel whose MTU is 1476 per default... 

The last point forces the router to drop all 1500-bytes packets and to send an ICMP message when a DF packet is 
received.

Hope it helps

- -eric

At 11:23 4/12/2003 +0000, marcel.cook () convergys com wrote:
> We have been suffering an issue to do with Checkpoint, Cisco GRE tunnels
> and MTU size for a number of months now, and I thought it might be worth
> posting a description of our problem on this list in case someone is able
> to help.  We feel that we have exhausted most avenues of trying to
> troubleshoot this issue.
>
> What we are trying to do is route Internet traffic for remote branch office
> sites via our central office's Internet connection.  As an example, we have
> a 2Mb AT&T Internet connection in our Paris office, connected to a 15Mb
> AT&T Internet connection in London.  We run a Cisco GRE tunnel between a
> 3640-VPN/MP router in Paris and a 7206VXR/G1 router in London.  In London,
> we also have a Nokia IP530 appliance running a fresh install of Check Point
> NG:AI, connected to a 10Mb PSINet Internet connection.
>
> The Cisco GRE tunnel has a MTU size of 1420 set at both ends for it's
> tunnel interfaces.  This is the highest we can use based on the
> encryption/encapsulation chosen in order to facilitate protocols such as
> OSPF from working over the link.  All other interfaces along the way
> (router ethernets and Nokia interfaces) are set the default 1500.
>
> The problem is that users in the Paris branch office are unable to view
> _some_ websites.  Examples of ones that don't work are www.yahoo.fr and
> www.adp.fr.  The majority work fine, including www.cisco.com and
> www.google.com.
>
> When running a tcpdump on the IP530 in London (on the external interface),
> during a session from Paris to one of the offending websites, the following
> is logged:
>
> 16:36:21.025051 O 154.38.47.5 > 194.3.182.10: icmp: 154.38.47.5 unreachable
> - need to frag (mtu 1420)
> 16:36:27.586541 I 194.3.182.10.80 > 154.38.47.5.41571: . 1:1461(1460) ack
> 249 win 63992 (DF)
> 16:36:27.588356 O 154.38.47.5 > 194.3.182.10: icmp: 154.38.47.5 unreachable
> - need to frag (mtu 1420)
> 16:36:40.711277 I 194.3.182.10.80 > 154.38.47.5.41571: . 1:1461(1460) ack
> 249 win 63992 (DF)
> 16:36:40.713043 O 154.38.47.5 > 194.3.182.10: icmp: 154.38.47.5 unreachable
> - need to frag (mtu 1420)
>
> We have also noticed that the packet size of traffic received from
> offending sites seems to be 1514 bytes.  For sites that work, i.e.
> cisco.com, it seems to be 1486 bytes.
>
> We have tried lots of things on the GRE tunnel configuration on our Cisco
> routers, including settings to ignore the Don't Fragment (DF) bit, and to
> force different MTU sizes.  A long-running Cisco TAC case has not suggested
> any way around our problem.
>
> Can anyone explain the cause of this problem, and suggest anything that can
> be done on our Nokia/Check Point configuration to prevent this occurring?
> Out of interest, when we route the Internet traffic past the Nokia IP530
> firewall and onto an Internet connection at another downstream site, which
> uses a Cisco PIX firewall instead, the remote Paris users ARE able to
> browse the offending websites successfully.  This indicates that it must be
> something to do with the Nokia/Check Point installation.
>
> Any comments or suggestions would be greatly received.
>
> Thanks,
> Marcel
>
> --
> "NOTICE:  The information contained in this electronic mail transmission is
> intended by Convergys Corporation for the use of the named individual or
> entity to which it is directed and may contain information that is
> privileged or otherwise confidential.  If you have received this electronic
> mail transmission in error, please delete it from your system without
> copying or forwarding it, and notify the sender of the error by reply email
> or by telephone (collect), so that the sender's address records can be
> corrected."
>
>
> _______________________________________________
> firewall-wizards mailing list
> firewall-wizards () honor icsalabs com
> http://honor.icsalabs.com/mailman/listinfo/firewall-wizards 

-----BEGIN PGP SIGNATURE-----
Version: PGP 7.0.1

iQA/AwUBP97YVXf6Gi0w+el5EQJK5wCgnsOtBbzRg0RDjyOB1U37FhiSbX8An205
zmYqvZoRIJlt1PsTO0tASXYu
=gTmt
-----END PGP SIGNATURE-----

_______________________________________________
firewall-wizards mailing list
firewall-wizards () honor icsalabs com
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards