Re: Announcing Project BISMark: ISP Performance Measurements from Home Routers

[Nick Feamster <feamster () cc gatech edu>]

Thanks for the feedback!
On Jun 28, 2011, at 6:13 AM, Mikael Abrahamsson wrote:

> On Mon, 27 Jun 2011, Nick Feamster wrote:
>
> > We've launched Project BISMark, a project that performs active performance measurements of upload and download 
> > throughput, latency, etc. from OpenWRT-based routers running inside of homes.  We have tested our OpenWRT image on 
> > the NetGear WNDR 3700v2 and are currently shipping out NetGear routers with the BISMark firmware to anyone who is 
> > interested.
>
> Please, pretty please, with sugar on top, don't just do active measurement, but also do passive measurement of real 
> traffic. Doing test traffic is one case, but the really important thing to look at is real traffic. I tried to get 
> traction for this on IETF75, but there seems to be little interest.

We would very much like to.  There are a number of reasons that regular users seem to be asking for passive 
measurement, such as monitoring of traffic usage of different applications (e.g., "How much is streaming eating into my 
usage cap?").  A few years ago, we had a tool that would do all of this with passive measurement 
(http://gtnoise.net/nano/), and we'd certainly like to resume this line of inquiry, if we can figure out how to address 
people's privacy concerns.  We are developing a passive measurement suite for BISMark, which is also available on 
github.

> On a NAT router there is a state table, what would the performance penality be to look at TCP sequence numbers, RTTs 
> (TCP timestamps) to be able to discern PDV and loss of the actual traffic the customer is doing?
>
> There are a lot of test suites, they solve one problem, but a passive monitoring system that would show how the real 
> traffic is behaving would yield a lot more valuable information that just relying on active testing (which will cause 
> harm to customer traffic when the test is run).

Definitely good points, and we've thought about this, for sure.  The key question seems to be how to handle user 
privacy in a way that everyone can be happy with.  Ultimately, we might take a survey of our users about this (e.g., 
certain people have said they don't mind tracking application performance/usage as long as the specific Web sites or 
destinations are not logged).  It would be really helpful to get an understanding of what users might find acceptable, 
as far as passive measurements.

-Nick