RE: CenturyLink RCA?

["Naslund, Steve" <SNaslund () medline com>]

I agree 100%.  Now they need to figure out why bricking the management network stopped forwarding on the optical side.  
> (Forgive my top posting, not on my desktop as I’m out of town)

Steven Naslund
Chicago IL
> Wild guess, based on my own experience as a NOC admin/head of operations at a large ISP - they have an automated 
> deployment system for new firmware for a (mission critical) piece of backbone hardware.
>
> They may have tested said firmware on a chassis with cards that did not exactly match the hardware they had in actual 
> deployment (ie: card was older hw revision in deployed hardware), and while it worked fine there, it proceeded >shit 
> the bed in the production.
>
> Or, they missed a mandatory low level hardware firmware upgrade that has to be applied separately before the other 
> main upgrade.
>
> Kinda picturing in my mind that they staged all the updates, set a timer, staggered reboot, and after the first hit 
> the fan, they couldn’t stop the rest as it fell apart as each upgraded unit fell on its own sword on reboot.
>
> I’ve been bit by the ‘this card revision is not supported under this platform/release’ bug more often then I’d like to 
> admit.
>
> And, yes, my eyes did start to get glossy and hazy the more I read their explanation as well.  It’s exactly the kind 
> of useless post I’d write when I want to get (stupid) people off my back about a problem.