Re: The FCC is planning new net neutrality rules. And they could enshrine pay-for-play. - The Washington Post

[Rick Astley <jnanog () gmail com>]

Here is a quote I made in the other thread around the same time you were
sending this:

"I also think the practice of paying an intermediary ISP a per Mbps rate in
order to get to a last mile ISP over a settlement free agreement is also a
bit disingenuous in cases where the amount of traffic is sufficient enough
to fill multiple links. Theoretically there are many times where the
intermediary ISP can hand off the traffic to a last mile ISP in exactly the
same building they received it in so they have very few of the costs of
actually delivering the traffic yet are the only party receiving money from
the content provider for delivery. This arrangement makes sense when the
traffic to the last mile ISP is a percentage of one link but after enough
links are involved the intermediary ISP is serving no real other purpose
than as a loophole used to circumvent paid peering fees (right or wrong)."

I think we are in agreement that $EyeballNetwork's customers pay it for
internet access and $ContentProvider should pay for their own pipes. But
where we diverge is with $CheapTransitProvider.

At least for the purpose of traffic following the path of $ContentProvider
> $CheapTransitProvider > $EyeballNetwork's because there is so much
traffic involved the only real purpose of the relationship with
$CheapTransitProvider is a loophole to get around paying $EyeballNetwork.
They are able to charge ridiculously low delivery prices because traffic is
only on their network for just long enough to say it touched and should now
be considered settlement free. It's little more than a cheap trick and it
makes them sort of the Cash4Gold of the Internet. I can completely
understand why $EyeballNetwork would tell $CheapTransitProvider they no
longer choose to have a settlement free agreement and they must buy future
ports.




On Sun, Apr 27, 2014 at 11:52 PM, Hugo Slabbert <hslabbert () stargate ca>wrote:

> Apologies that I dropped offlist as I was out for the day.  I think the
> bulk of my thoughts on this have already been covered by others since,
> including e.g. Matt's poor grandmother and her phone dilemma in the "What
> Net Neutrality should and should not cover" thread.
>
> Basically I think we're on the same page for the most part, with maybe
> some misunderstandings between us.
>
> > I covered this scenario in more detail in my post "What Net Neutrality
> should and should not cover" but if you expand on the assumption that
> paying for an internet connection also pays for the direct connection of
> every party who you exchange traffic with then you have a scenario where
> only half the people connected to the Internet should have to pay at all
> for their connection because any scenario where people simply buy their own
> pipe would be considered "double billing".
>
> I don't think anyone on the Netflix^H^H^H^H^H^H $ContentProvider side of
> this was saying that $ContentProvider should get everything handed to them
> on a silver platter.  $ContentProvider pays for transit sufficient to
> handle the traffic that their customers request.  $EyeballNetwork's
> customers pay it for internet access, i.e. to deliver the content that they
> request, e.g. from $ContentProvider.  That covers both directions here.
>  Links between $ContentProvider's transit provider and $EyeballNetwork were
> getting congested, and $EyeballNetwork refuses to upgrade capacity.  Where
> we were getting into the double-dip was $EyeballNetwork saying to
> $ContentProvide:  "Hey, we know you already pay for transit, but you're
> gonna have to pay us as well if you want us to actually accept the traffic
> our customers requested".
>
> The alternate arrangement between $ContentProvider and $EyeballNetwork
> seems to be private peering, where again it would seem to be fair for each
> side to bring the needed transport and ports to peering points.  In recent
> history, though, it seems that $EyeballNetwork came out ahead in that
> agreement somehow.  Now, Tore brought up a good point on paid peering in
> cases where e.g. $EyeballNetwork is already exchanging traffic with
> $ContentProvider through existing peering or below their CDR on existing
> transit, and indeed it seems that was the case for $EyeballNetwork via
> peering with $CheapTransitProvider that $ContentProvider was using.  But it
> seems that $EyeballNetwork was having a pissing match with
> $CheapTransitProvider and refusing to upgrade ports.
>
> "Okay", says $ContentProvider.  "How about we just peer directly."
> "Sounds great," says $EyeballNetwork.  "Since we have to allocate capacity
> for this discrete from our existing peering capacity, you'll need to foot
> the bill for that."
> "Huh?" says $ContentProvider.  "This could have been fixed by you
> increasing your peering capacity to match the traffic volume your users are
> requesting, but you didn't want to do that because of your tiff with
> $CheapTransitProvider.  Tell me again why we're paying for your side of
> this *in addition* to our own when we're only going this route because of a
> decision *you* made?"
> "Because you need to reach our customers, and we're the only path to them,
> so we have leverage."
> *blank stare*
> "So you're willing to give your customers crappy service because your
> customers don't have alternate options and you think we need this more than
> you do?"
> "That's a possibility."
> "I hate you."
> "I know; sign here please."
>
> But, again, this is outside looking in.  For now, I'll pick up a copy of
> Bill Norton's Internet Peering book as per Bob's suggestion, for some light
> Sunday night reading.
>
> Cheers,
>
> --
> Hugo
>
> ________________________________
> From: Rick Astley <jnanog () gmail com>
> Sent: Sunday, April 27, 2014 8:45 AM
> To: Hugo Slabbert
> Cc: nanog () nanog org
> Subject: Re: The FCC is planning new net neutrality rules. And they could
> enshrine pay-for-play. - The Washington Post
>
> If it were through a switch at the exchange it would be on each of them to
> individually upgrade their capacity to it but at the capacities they are at
> it they are beyond what would make sense financially to go over an exchange
> switch so they would connect directly instead. It's likely more along the
> lines of needing several 100G ports as Netflix is over 30% of peak usage
> traffic in North America:
>
> "Netflix (31.6%) holds its ground as the leading downstream application in
> North America and together with YouTube (18.6%) accounts for over 50% of
> downstream traffic on fixed networks."  (source
> https://www.sandvine.com/trends/global-internet-phenomena/ )
>
> That amount of data is massive scale. I don't see it as double dipping
> because each party is buying the pipe they are using. I am buying a 15Mbps
> pipe to my home but just because we are communicating over the Internet
> doesn't mean the money I am paying covers the cost of your connection too.
> You must still buy your own pipe in the same way Netflix would. I covered
> this scenario in more detail in my post "What Net Neutrality should and
> should not cover" but if you expand on the assumption that paying for an
> internet connection also pays for the direct connection of every party who
> you exchange traffic with then you have a scenario where only half the
> people connected to the Internet should have to pay at all for their
> connection because any scenario where people simply buy their own pipe
> would be considered "double billing".
>
> The cost for residential broadband is high enough in the US without a
> policy like that in place. If there is one policy that would keep poor
> families from being able to afford broadband it would be that one.
>
>
>
>
>
> On Sun, Apr 27, 2014 at 2:58 AM, Hugo Slabbert <hslabbert () stargate ca
> <mailto:hslabbert () stargate ca>> wrote:
>
> > ...but if that point of congestion is the links between Netflix and
> Comcast...
>
> Which, from the outside, does appear to have been the case.
>
> > ...then Netflix would be on the hook to ensure they have enough capacity
> to Comcast to get the data at least gets TO the Comcast network.
>
> Which I don't believe was a problem?  Again, outside looking in, but the
> appearances seemed to indicate that Comcast was refusing to upgrade
> capacity/ports, whereas I didn't see anything indicating that Netflix was
> doing the same.  So:
> > I have gear; you have gear.  I upgrade or add ports on my side; you
> upgrade or add ports on your side.
>
>
> > The argument at hand is if Comcast permitted to charge them for the
> links to get to their network or should they be free/settlement free. I
> think it should be OK to charge for those links as long as its a fair
> market rate and the price doesn't basically amount to extortion.
>
> Are we talking here about transport between Netflix's POPs and Comcast's?
>  I definitely don't expect Comcast to foot the bill for transport between
> the two, and if Netflix was asking for that I'm with you that would be out
> of line.  If there are existing exchange points, though, would it not be
> reasonable to expect each side to up their capacity at those points?
>
>
> > Once that traffic is given directly to comcast no other party receives
> payment for delivering it so there is no double billing.
>
> The "double-dip" reference was to charging both the content provider and
> the ISP's own customer to deliver the same bits.  If the traffic from
> Netflix was via Netflix's transit provider and Comcast then again was
> looking to bill Netflix to accept the traffic, we'd hit double billing.
>
> I guess that's the question here:  If additional transport directly been
> POPs of the two parties was needed, somebody has to pay for the links.
>  Releases around the deal seemed to indicate that the peering was happening
> at IXs (haven't checked this thoroughly), so at that point it would seem
> reasonable for each party to handle their own capacity to the peering
> points and call it even.  No?
>
> --
> Hugo
>
> ________________________________
> From: Rick Astley <jnanog () gmail com<mailto:jnanog () gmail com>>
> Sent: Saturday, April 26, 2014 11:23 PM
> To: Hugo Slabbert
> Cc: nanog () nanog org<mailto:nanog () nanog org>
> Subject: Re: The FCC is planning new net neutrality rules. And they could
> enshrine pay-for-play. - The Washington Post
>
> > How is this *not* Comcast's problem?  If my users are requesting more
> traffic than I banked on, how is it not my responsibility to ensure I have
> capacity to handle that?  I have gear; you have gear.  I upgrade or add
> ports on my side; you upgrade or add ports on your side.  Am I missing
> something?
>
> Sort of yes, it's Comcasts problem to upgrade subscriber lines but if that
> point of congestion is the links between Netflix and Comcast then Netflix
> would be on the hook to ensure they have enough capacity to Comcast to get
> the data at least gets TO the Comcast network. The argument at hand is if
> Comcast permitted to charge them for the links to get to their network or
> should they be free/settlement free. I think it should be OK to charge for
> those links as long as its a fair market rate and the price doesn't
> basically amount to extortion. Sadly the numbers are not public so I
> couldn't tell you one way or the other aside from I disagree with the
> position Netflix seems to be taking that they simply must be free. Once
> that traffic is given directly to comcast no other party receives payment
> for delivering it so there is no double billing.
>
> This diagram best describes the relationship (ignoring pricing):
> http://www.digitalsociety.org/files/gou/free-and-paid-peering.png
>
> "Content provider" would be Netflix and Comcast would be Broadband ISP 1.
>
>
>
>
> On Sun, Apr 27, 2014 at 1:56 AM, Hugo Slabbert <hslabbert () stargate ca
> <mailto:hslabbert () stargate ca><mailto:hslabbert () stargate ca<mailto:
> hslabbert () stargate ca>>> wrote:
> Okay, I'm not as seasoned as a big chunk of this list, but please correct
> me if I'm wrong in finding this article a crock of crap.  With
> Comcast/Netflix being in the mix and by association Cogent in the
> background of that there's obviously room for some heated opinions, but
> here goes anyway...
>
> > A long, long time ago when the Internet was young and few, if any had
> thought
> > to make a profit off it, an unofficial system developed among the network
> > providers who carried the traffic: You carry my traffic and I'll carry
> yours
> > and we don't need money to change hands. This system has collapsed under
> > modern realities.
>
> I wasn't aware that settlement-free peering had "collapsed".  Not saying
> it's the "only way", but "she ain't dead yet".
>
> Seltzer uses that to set up balanced ratios as the secret sauce that makes
> settlement-free peering viable:
> "The old system made sense when the amount of traffic each network was
> sending to the other was roughly equivalent."
>
> ...and since Netflix sends Comcast more than it gets, therefor Netflix
> needs to buck up:
> "Of course Netflix should pay network providers in order to get the huge
> amounts of bandwidth they require in order to reach their customers with
> sufficient quality."
>
> But this isn't talking about transit; this is about Comcast as an edge
> network in this context and Netflix as a content provider sending to
> Comcast users the traffic that they requested.  Is there really anything
> more nuanced here than:
>
> 1.  Comcast sells connectivity to their end users and sizes their network
> according to an oversubscription ratio they're happy with.  (Nothing wrong
> here; oversubscription is a fact of life).
> 2.  Bandwidth-heavy applications like Netflix enter the market.
> 3.  Comcast's customers start using these bandwidth-heavy applications and
> suck in more data than Comcast was betting on.
> 4.  Comcast has to upgrade connectivity, e.g. at peering points with the
> heavy inbound traffic sources, accordingly in order to satisfy their
> customers' usage.
>
> How is this *not* Comcast's problem?  If my users are requesting more
> traffic than I banked on, how is it not my responsibility to ensure I have
> capacity to handle that?  I have gear; you have gear.  I upgrade or add
> ports on my side; you upgrade or add ports on your side.  Am I missing
> something?
>
> Overall it seems like a bad (and very public) precedent & shift towards
> double dipping, and the pay-for-play bits in the bastardized "Open
> Internet" rules don't help on that front.  Now, Comcast is free to leverage
> their customers as bargaining chips to try to extract payments, and Randy's
> line of encouraging his competitors to do this sort thing seems fitting
> here.  Basically this doesn't harm me directly at this point.  Considering
> the lack of broadband options for large parts of the US, though, it seems
> that end users are getting the short end of the stick without any real
> recourse while that plays out.
>
> --
> Hugo
>
> ________________________________________
> From: NANOG <nanog-bounces () nanog org<mailto:nanog-bounces () nanog org
> > <mailto:nanog-bounces () nanog org<mailto:nanog-bounces () nanog org>>> on
> behalf of Larry Sheldon <LarrySheldon () cox net<mailto:LarrySheldon () cox net
> > <mailto:LarrySheldon () cox net<mailto:LarrySheldon () cox net>>>
> Sent: Saturday, April 26, 2014 4:58 PM
> To: nanog () nanog org<mailto:nanog () nanog org><mailto:nanog () nanog org<mailto:
> nanog () nanog org>>
> Subject: Re: The FCC is planning new net neutrality rules. And they could
> enshrine pay-for-play. - The Washington Post
>
> h/t Suresh Ramasubramanian
>
> FCC throws in the towel on net neutrality
>
> http://www.zdnet.com/fcc-throws-in-the-towel-on-net-neutrality-7000028770/
>
> Forward!  On to the next windmill, Sancho!
> --
> Requiescas in pace o email           Two identifying characteristics
>                                          of System Administrators:
> Ex turpi causa non oritur actio      Infallibility, and the ability to
>                                          learn from their mistakes.
>                                            (Adapted from Stephen Pinker)