nanog mailing list archives

RE: qwest outage no notice


From: "Richey" <mylists () battleop com>
Date: Thu, 7 Jan 2010 12:12:00 -0500

That should read...

Yeah, they refused to notify due to [marketing] concerns from what they


Richey



-----Original Message-----
From: Mike [mailto:mike-nanog () tiedyenetworks com] 
Sent: Thursday, January 07, 2010 12:04 PM
Cc: nanog () nanog org
Subject: Re: qwest outage no notice


Yeah, they refused to notify due to security concerns from what they 
told me last night. Notification was performed after maintenance was 
complete.


Ok, so the next question is, what harm would a simple advance notice of
'emergency maintenance' caused, vs the very real hassle and inconvenience
that DS3-down in the middle of the night caused for operations staff? I
personally was yanked out of bed by my network monitoring systems and had to
spend at least 5 minutes looking at logs and perf monitors and so forth so I
had enough information before making the call to qwest support, thinking we
took a hit on our fiber. Some others reported getting 7 or more hours of
advance notice - this would have been enough for me, even tho of course I
don't like it - and I suspect it could have saved many many others from a
similar fate of responding to the event so late.

Just saying "we need to do this", does not give the bad guys any ammo with
which to attack, and would go a long ways twords preventing needless heroics
such as middle of the night investigations by senior staff. The follow up
email we subsequently received was fine and we appreciated learning it
really was a necessary upgrade and seems justified now with that knowledge,
I would simply have appreciated not having to engage my emergency processes
for something that was planned.

Mike-




Current thread: