Re: Long hops on international paths

[Dave Cohen <craetdave () gmail com>]

I guess it depends what you’re considering a “very few” number of routers but this seems to be an expected outcome. 
While there are a large number of wet cable landing stations, they are highly concentrated near a small number of metro 
areas, and with the exception of capacity owned by the ILECs, the supermajority of routers terminating that capacity in 
the US are going to live in fewer than ten discrete carrier hotel locations. (It’s worth noting that terrestrial 
capacity coming in from Mexico and Canada also terminates in a small number of locations, although the overlap between 
the two lists is fairly small). In addition, while the links likely terminate in multiple devices at a given location, 
carriers more likely to undersubscribe transoceanic core capacity than other areas of the core, which means that for 
many carriers it’s unlikely you’d see multiple paths show up in a trace unless you catch it during an outage situation. 
That said, seeing transoceanic links terminate in Chicago is likely an artifact of hops missing in a trace; although I 
am familiar with a couple of more niche providers that extend transoceanic capacity into non-coastal markets on optical 
gear in order to meet specific performance needs, this is unlikely to be seen in the network of a Tier 1 or similarly 
scaled network. 

Dave Cohen
craetdave () gmail com

> On Jan 17, 2022, at 6:15 PM, Christopher Morrow <morrowc.lists () gmail com> wrote:
>
> 
>
>
> > On Mon, Jan 17, 2022 at 5:31 PM PAUL R BARFORD <pb () cs wisc edu> wrote:
> > Dear Pengxiong,
> >
> > Thanks for your questions:
> >
> > We are using CAIDA’s Internet Topology Data Kit (ITDK) that uses the MIDAR alias resolution method to infer IP 
> > addresses assigned to the same router.
> > We understand the concerns about IP geolocation.  Interfaces of the router in question are assigned similar domain 
> > names e.g., “chi-b2-link.ip.twelve99.net” (62.115.50.61). We also used CAIDA’s ITDK, which provides geolocation 
> > information, and indicates that this router is located in Chicago.  We cross-reference with Maxmind where possible.  
> > In this particular case, there is the telltale in the use of "chi" in the domain name. 
>
> I think nick's point about ttl expiry and missing some context on topology still stands.
> I'd be that the paths between 2 continents do not actually land in chicago... that you're seeing (or not seeing) 
> missing hops between the coast(s) and chicago inside 1299's network in the US.
>  
> > Hope that helps.
> >
> > Regards, PB
> > From: Pengxiong Zhu <pzhu011 () ucr edu>
> > Sent: Monday, January 17, 2022 3:23 PM
> > To: PAUL R BARFORD <pb () cs wisc edu>
> > Cc: nanog () nanog org <nanog () nanog org>
> > Subject: Re: Long hops on international paths
> >  
> > Hi Paul,
> >
> > Just curious. How do you determine they are the same routers? Is it based on IP address or MAC addresses? Or using 
> > CAIDA’s router alias database?
> >
> > Also how do you draw the conclusion that the AS1299 router is indeed in Chicago? IP-geolocation based on rDNS is not 
> > always accurate though. 
> >
> >
> > Pengxiong 
> >
> > On Mon, Jan 17, 2022 at 10:03 AM PAUL R BARFORD <pb () cs wisc edu> wrote:
> > Hello,
> >
> > I am a researcher at the University of Wisconsin.  My colleagues at Northwestern University and I are studying 
> > international Internet connectivity and would appreciate your perspective on a recent finding.
> >
> > We're using traceroute data from CAIDA's Ark project for our work.  We've observed that many international links 
> > (i.e., a single hop on an end-to-end path that connects two countries where end points on the hop are identified via 
> > rDNS) tend to originate/terminate at the same routers.  Said another way, we are observing a relatively small set of 
> > routers in different countries tend to have a majority of the international connections - this is especially the 
> > case for hops that terminate in the US.  For example, there is a router operated by Telia (AS1299) in Chicago that 
> > has a high concentration of such links.  We were a bit surprised by this finding since even though it makes sense 
> > that the set of providers is relatively small (i.e., those that offer global connectivity), we assumed that the set 
> > of routers that used for international connectivity within any one country would tend to be more widely distributed 
> > (at least with respect to how they appear in traceroute data - MPLS notwithstanding).
> >
> > We're interested in whether or not this is indeed standard practice and if so, the cost/benefit for configuring 
> > international connectivity in this way?
> >
> > Any thoughts or insights you might have would be greatly appreciated - off-list responses are welcome.
> >
> > Thank you.
> >
> > Regards, PB
> >
> > Paul Barford
> > University of Wisconsin - Madison
> >
> > -- 
> >
> > Regards,
> > Pengxiong Zhu
> > Department of Computer Science and Engineering
> > University of California, Riverside