nanog mailing list archives
Re: Best utilizing fat long pipes and large file transfer
From: Robert Boyle <robert () tellurian com>
Date: Thu, 12 Jun 2008 19:26:56 -0400
At 06:37 PM 6/12/2008, you wrote:
I'm looking for input on the best practices for sending large files over a long fat pipe between facilities (gigabit private circuit, ~20ms RTT). I'd like to avoid modifying TCP windows and options on end hosts where possible (I have a lot of them). I've seen products that work as "transfer stations" using "reliable UDP" to get around the windowing problem.I'm thinking of setting up servers with optimized TCP settings to push big files around data centers but I'm curious to know how others deal with LFN+large transfers.
In our experience, you can't get to line speed with over 20-30ms of latency using TCP regardless of how much you tweak it. We transfer files across the US with 60-70ms at line speeds with UDP based file transfer programs. There are a number of open source projects out there designed for this purpose.
-Robert Tellurian Networks - Global Hosting Solutions Since 1995 http://www.tellurian.com | 888-TELLURIAN | 973-300-9211 "Well done is better than well said." - Benjamin Franklin
Current thread:
- Re: Best utilizing fat long pipes and large file transfer, (continued)
- Re: Best utilizing fat long pipes and large file transfer Randy Bush (Jun 12)
- Message not available
- Re: Best utilizing fat long pipes and large file transfer Randy Bush (Jun 12)
- Re: Best utilizing fat long pipes and large file transfer Darren Bolding (Jun 12)
- Re: Best utilizing fat long pipes and large file transfer Robert E. Seastrom (Jun 12)
- Re: Best utilizing fat long pipes and large file transfer Kevin Oberman (Jun 12)
- Re: Best utilizing fat long pipes and large file transfer Robert E. Seastrom (Jun 13)
- Re: Best utilizing fat long pipes and large file transfer Randy Bush (Jun 12)
- Re: Best utilizing fat long pipes and large file transfer Glen Turner (Jun 15)
- Re: Best utilizing fat long pipes and large file transfer Robert E. Seastrom (Jun 16)
- Re: Best utilizing fat long pipes and large file transfer Kevin Oberman (Jun 12)
- Re: Best utilizing fat long pipes and large file transfer Kevin Oberman (Jun 13)
- Re: Best utilizing fat long pipes and large file transfer Robert Boyle (Jun 13)
- Re: Best utilizing fat long pipes and large file transfer Deepak Jain (Jun 13)
- Re: Best utilizing fat long pipes and large file transfer goemon (Jun 13)
- Re: Best utilizing fat long pipes and large file transfer Glen Turner (Jun 15)
- Re: Best utilizing fat long pipes and large file transfer goemon (Jun 16)
- Re: Best utilizing fat long pipes and large file transfer Bob Bradlee (Jun 16)
- Re: Best utilizing fat long pipes and large file transfer Jay R. Ashworth (Jun 22)