Re: mysterious packet delay to/from www.caida.org was: Cisco Netflow Analysis Software

["Yu Ning" <yuning () cndata com>]

Hi,

It's very interesting that i encountered this problem yesterday. If i'm
not mistaken: you have different source address block which have different
trace delay to the same external site? The huge jump of delay is up to
the source address, not the host OS. Right ?

So i think it may be the reason of asymmetric routing. In more detail, 
say we you have source addr. A, B, and external destination C. Your trace
A->C has a larger delay than B->C . The reason is that the backward path from C->A
is different than C->B (it's very possible, because you, or your upstream
may advertise add block A, B differently), which has a larger delay.

You can verify this in this way. Let's A, B trace to a same external 
traceroute server address (say net.yahoo.com, or others). You may
find they have the same OUTBOUND path. But then trace back from the
traceroute server to A, and B, i believe you will find the difference.

hope this help, regards.

Yu Ning


--------------------------------------
(Mr.) Yu Ning 
ChinaNet Operation Center
Networking Dep.,Datacom Bureau
China Telecom.,Beijing(100088),P.R.C
+86-10-82078519/62359464/62367444(fax)
--------------------------------------
----- Original Message ----- 
From: Kai Schlichting <kai () pac-rim net>
To: <nanog () merit edu>
Cc: <info () caida org>; <kai () pac-rim net>
Sent: Thursday, March 16, 2000 5:24 AM
Subject: mysterious packet delay to/from www.caida.org was: Cisco Netflow Analysis Software


> Can anyone else confirm the following observation ?
> If yes: what might be causing this ? (CEF is my usual suspect, but toggling it
> on/off doesn't do anything)
>
> Naturally, I went to www.caida.org in response to one of the NANOG postings,
> but seemingly hit a roadblock: the site just wouldnt want to load properly.
> As engineers from time to time get bitten by the curiosity bug, I tracerouted
> and ping'd around to find the cause of this non-responsiveness.
>
> Surprise: depending on the IP space used to access the site, there would be
> a considerable pause in the packet flow: 20 sec. for a BSDI 4.1 machine or
> a Win98 machine, 80 sec. for an OpenBSD 2.4 machine , but NO pause when using a
> different set of IP space (to be exact: Interface IP space belonging to
> upstream provider). One could literally switch around the 'ip telnet
> source-interface' on the gateway router to use the different IP space blocks,
> and would get the long delay or not, depending on which space was used.
>
> Note that there is no duplicates or visible packet loss whatsoever,
> and the delay is occurring at the exact same packet in the flow each
> and every time, regardless of client operating system (tcp receive
> buffers/window are set to 2^16 though, which didn't make a difference
> other than different packetization).
>
> I have tried this from 2 other locations, and the long pause (there) would
> melt down into the 100ms range mostly sometimes go away.
>
> Sorry for the tcpdump in advance...a simple GET / HTTP/1.0 on their server.
> This is out of 208.192.0.0/10 (AS 701) , btw., with no more specific route
> visible net-wide.
>
> bye,Kai
>
> ------ (BSDI 4.x at work here:)
> 15:35:57.505400 host.35570 > ipn.caida.org.www: S 3494476576:3494476576(0) win 65535 <mss 
> 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,[|tcp]>
> [tos 0x10] (ttl 64, id 5418)
> 15:35:57.700158 ipn.caida.org.www > host.35570: S 1151006778:1151006778(0) ack 3494476577 win 8760 <mss 1460> (DF) 
> (ttl 239, id 11105)
> 15:35:57.700794 host.35570 > ipn.caida.org.www: . ack 1 win 65535 [tos 0x10] (ttl 64, id 12399)
> 15:36:01.446201 host.35570 > ipn.caida.org.www: P 1:17(16) ack 1 win 65535 [tos 0x10] (ttl 64, id 24844)
> 15:36:01.714702 ipn.caida.org.www > host.35570: . ack 17 win 8760 (DF) (ttl 239, id 11106)
> 15:36:01.715261 host.35570 > ipn.caida.org.www: P 17:19(2) ack 1 win 65535 [tos 0x10] (ttl 64, id 463)
> 15:36:01.972221 ipn.caida.org.www > host.35570: P 1:1461(1460) ack 19 win 8760 (DF) (ttl 239, id 11107)
> 15:36:01.972910 host.35570 > ipn.caida.org.www: . ack 1461 win 64240 [tos 0x10] (ttl 64, id 10930)
> 15:36:02.240938 ipn.caida.org.www > host.35570: . 1461:2921(1460) ack 19 win 8760 (DF) (ttl 239, id 11108)
> 15:36:02.246988 ipn.caida.org.www > host.35570: P 2921:4097(1176) ack 19 win 8760 (DF) (ttl 239, id 11109)
> 15:36:02.252069 host.35570 > ipn.caida.org.www: . ack 4097 win 65535 [tos 0x10] (ttl 64, id 24582)
> 15:37:22.984047 ipn.caida.org.www > host.35570: P 4097:4933(836) ack 19 win 8760 (DF) (ttl 239, id 30829)
> 15:37:22.984192 ipn.caida.org.www > host.35570: F 4933:4933(0) ack 19 win 8760 (DF) (ttl 239, id 30830)
> 15:37:22.987340 host.35570 > ipn.caida.org.www: . ack 4934 win 64864 [tos 0x10] (ttl 64, id 53067)
> 15:37:22.999506 host.35570 > ipn.caida.org.www: F 19:19(0) ack 4934 win 65535 [tos 0x10] (ttl 64, id 36125)
> 15:37:23.222742 ipn.caida.org.www > host.35570: . ack 20 win 8760 (DF) (ttl 239, id 30831)