Re: [pfSense Support] Strange TCP connection behavior 2.0 RC2 (+3)

[Jérôme Nicolle <jerome () ceriz fr>]

This is a well known issues called "Long Fat Pipe Network".

There's many university papers about it and many tricks to get around
it on software-based boxes.

Adjusting your TCP window size was the best start, if it's set
properlu. The basic formula is provided in this forum post :
http://forums.whirlpool.net.au/archive/116165

Good luck !

2011/6/28 Eugen Leitl <eugen () leitl org>:
> ----- Forwarded message from William Salt <williamejsalt () googlemail com> -----
>
> From: William Salt <williamejsalt () googlemail com>
> Date: Tue, 28 Jun 2011 08:03:25 +0100
> To: support () pfsense com
> Subject: [pfSense Support] Strange TCP connection behavior 2.0 RC2 (+3)
> Reply-To: support () pfsense com
>
> Hi All,
>         For the last couple of months i have been pulling my hair out
> trying to solve this problem.
> We have a 1Gbps transatlantic link from the UK to the US, which has
> successfully passed the RFC2544 test.
>
> At either end, we have a media converter, and a supermicro server with an
> intel quad port NIC running pfsense 2 (RC2 at one end RC3 at the other) and
> the IGB driver on the quad port.
>
> We can pass 1gbps either way with UDP. However we are experiencing very
> strange issues with tcp connections.
>
> With window scaling enabled, and a max socket buffer set to 16MB, we see no
> difference.
> Even disabling window scaling and setting the window to 16MB makes no
> difference.
>
> Each TCP connection starts very slowly, and will max out at around 190mbps,
> taking nearly 2 minutes to climb to this speed before *plateauing*.
>
> We have to initiate many (5+) connections to saturate the link with tcp
> connections with iperf.
>
> Real world tests transferring files, max out at 100mbps, using multiple
> connections.
>
> I have followed guides like this:
> http://www.psc.edu/networking/projects/tcptune/#FreeBSD
>
> With no luck, and have tweaked, disabled, and enabled nearly every relevant
> sysctl parameter with no luck.
>
> Can anyone shed some light on this?
>
> I am now doubting the IGB driver, and am looking to swap out the cards as a
> last ditch effort.
> However, we have tried different hardware (L3 switches, media convertes +
> laptops etc), and the symptoms still persist...
> The only constant is freebsd 8.1 - pfsense (or 8.2 for our production
> systems).
> I have tried the freebsd net mailinglist, but im hoping you lot can help me!
>
> Cheers in advance
> Will
>
> ----- End forwarded message -----
> --
> Eugen* Leitl <a href="http://leitl.org";>leitl</a> http://leitl.org
> ______________________________________________________________
> ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
> 8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE



-- 
Jérôme Nicolle
06 19 31 27 14