Re: more on savvis outtages...

[Deepak Jain <deepak () ai net>]

Depending on if the have a primary or "virtual" POP in Pittsburgh, a
single circuit can pull the entire VPOP down. From what I understand of
their network, they backhaul all circuits of a VPOP to a "real" pop. So if
they have a hardware, port or circuit failure on the single channelized
circuit, everything is gone. Also since they bring most of their circuits
(especially T-1) over a CT3, any hiccup in that will cause a problem too. 

I think the "entire Pittsburgh" network going down may be a function of a
small number of total circuits rather than a "massive failure" of
infrastructure equipment or services. 

My knowledge of Savvis is possibly dated and this is meant in no way as a
compliment or criticism of their network topology or operations.

Deepak Jain
AiNET

On Mon, 7 Aug 2000, deeann mikula wrote:

> pgh. lost savvis again around 6pm EDT.  i called then, and got a "don't
> really know yet..." response from their noc.
>
> after getting the msg below (8pm EDT), i called them back, and the noc
> tech told me that they are working at it from 2 angles, one group on
> the possibility of an equiptment failure and another group working
> with an at&t tech on the possibility that the problem lies with them.
> he said he "seems like" it might be a re-occurence of y'days problem,
> but it was too soon to tell and he hadn't gotten an official
> confirmation that it was ok to say that yet.
>
> IMHO, i think their handling of this (emails and prompt call backs,
> and informed and cheerful phone staff) has been great (from a customer
> support viewpoint.)  now if they could just get us back up again... 3
> separate several hour outages of the complete pgh. network is just
> unacceptable. :( [8am-2pm sunday, 4am-6am monday and 6pm-? monday.]
>
> just my update for all those of pgh net-admins, and anyone else who cares.
>
>
>      deeann m.m. mikula
>    director of operations
>
> telerama public access internet
>    http://www.telerama.com 
>
>
> ---------- Forwarded message ----------
> Date: Mon, 7 Aug 2000 18:23:15 -0500 (CDT)
> From: <snipped, from savvis noc>
>
> Subject: NOTIFICATION OF EMERGENCY OUTAGE
>
> At 8/7/2000 17:28:33 CST, we experienced an outage in our ATM and
> Frame Relay switches in Pittsburgh.  We have escalated this issue to
> the highest levels in Savvis, and have also opened tickets with
> carriers.  If you have any questions or concerns, please call the NCC
> at (888) 638-6771 and reference ticket 322411.