Re: FCC releases Internet speed test tool

[Fred Baker <fred () cisco com>]

On Mar 12, 2010, at 5:43 AM, Marshall Eubanks wrote:
> http://www.broadband.gov/


I'm listening to all this and thinking through the questions the FCC might be asking. I'm also trying to do a 
somewhat-controlled test, which I'll give you the first several samples of. See attached.

I picked up your note at ~7:10 PST this morning and set up some timed commands to remind myself to try this out once an 
hour at a few minutes before my various meetings start. I'm testing speakeasy against speedtest against the two 
broadband.gov engines, plus pingtest just for fun. I am of course at work today, woking from home.

For the record, I am a Cox Business subscriber, and my contract is 2 MBPS down and 384 KBPS up. That implies I'm not 
going see tens of MBPS, and I would be surprised if the numbers were significantly different than advertised as I am by 
definition paying more money for less service. Some of the tests will run in parallel with my daily workload, and I'll 
try to keep that straight. What may impinge is mail downloads, which happen under the hood and aren't necessarily 
visible at the time I initiate a test.

An observation on the various comments that "going to a test service operated somewhere other than my POP is a dumb 
idea": it depends on what you're measuring. If you're measuring, as I imagine those commentators are, what bit rate is 
available on the link between the residential subscriber and the ISP and therefore whether the contract is being met, 
the point is well taken. If the point is "what is a reasonable expectation of bandwidth when accessing various things 
on the Internet", the ISP's internal connectivity, connectivity to its upstream, and to its peers is also relevant - 
and from an FCC Net Neutrality perspective pretty important. A fairly common report several years ago was that on DSL 
networks one might get a high rate through the very last mile but often got mere tens of KBPS through the back end 
network, and DSL marketing made the same comment about Cable Modem networks. When I buy a certain rate from an ISP, the 
point is not to talk with the ISP at that rate; the point is to be able to do what I do, such as running a VPN across 
<ISP> and <upstream> to/from <company>, or access content on the web.

Another observation: when a subscriber buys a bit rate, the bit rate includes IP headers, link layer overhead, etc. If 
I use FTP to test my rate, it is measuring the rate at which TCP can deliver user data, which is to say that it omits 
the TCP, IP, and link layer overheads, which are on the order of 3-4% of the bandwidth. If I were running one of these 
tests over a circuit switch link such as a T-1, it would not measure that it was delivering 1.544 MBPS plus or minus 75 
ppm; it would measure somewhat less considering both physical layer overheads (2/193 gets lost out of a T-1 frame) and 
TCP/IP overheads.

What I have seen so far this morning is that speakeasy, speedtest, and the two broadband.gov sites come up with about 
the same numbers, modulo obvious issues of being different tests at slightly different times. The one difference there 
is with broadband.gov/MLAB: it seems to measure my upload rate at about half of contract rate the first time I test it, 
and then measure something approximating the contract if I repeat the test. No idea what that really means - if it 
randomly was high and low I could argue that it is a capacity-at-tester or "did POP download email?" issue, but since 
it always the first test that is low it suggests something relevant to the sequence.