Re: outages list down? Lightower, Worcester, MA fiber cut

[Jared Mauch <jared () puck nether net>]

RFO: underlying filesystem issues on the hypervisor caused the system to “freak out” (yes, that’s the technical term).  
After a reboot, some fighting with java iKVM the system is back up.  Sorry for the trouble.  

OT: (you can delete if you don’t care about my random stuff)

I have some bigger plans to start to split the various parts of puck into more distributed systems (e.g.: DNS services, 
mail lists, etc) of time, including the various open.*project sites and databases).

Last hardware refresh was in fall of 2011, so with the falling prices of SSD, etc it’s likely time to transition.

- Jared

> On Nov 7, 2014, at 7:34 AM, Chuck Anderson <cra () WPI EDU> wrote:
>
> I can't get to the mailing list page, and neither can
> downforeveryoneorjustme.com:
>
> http://downforeveryoneorjustme.com/puck.nether.net
>
> I've sent a couple new email messages there and they haven't been
> delivered either.
>
> Here is the latest message I sent regarding the Lightower, Worcester,
> MA fiber cut:
>
> Date: Fri, 7 Nov 2014 07:26:55 -0500
> From: Chuck Anderson <cra () WPI EDU>
> To: outages () outages org
> Subject: Re: [outages] Lightower, Worcester, MA fiber cut
>
> Update as of 06:05:16 -0500:
>
> "Status has remained the same Lightower has two (2) 288 cables that
> were burnt through in a National Grid  enclosure. National Grid has
> informed us that there is an exposed high tension power line in this
> enclosure and there will be no access granted in the near
> future. National Grid has deemed this manhole unsafe at this
> time. Lightower is working on an alternate solution that will re-route
> fibers around the affected man hole still NO ETTR at this time. We
> apologize for this inconvenience."
>
> On Fri, Nov 07, 2014 at 04:04:27AM -0500, Chuck Anderson wrote:
> > Updates as of 01:41:40 -0500 and 02:58:11 -0500:
> >
> > "We have two (2) 288 cables that were burnt through in this National
> > Grid enclosure. National Grid has informed us that there is an exposed
> > high tension power line in this enclosure and there will no access
> > granted in the near future. National Grid has deemed this manhole
> > unsafe at this time. Lightower is working on an alternate solution
> > that will re-route fibers around the affected man hole. NO ETTR at
> > this time."
> >
> > My initial outage time was exactly 18:58:54 EST on two circuits.
> >
> > On Thu, Nov 06, 2014 at 08:49:16PM -0500, Chuck Anderson via Outages wrote:
> > > We lost several dark fiber circuits out of Lightower in Worcester, MA.
> > > There is a manhole fire down the street at Main St. & Highland
> > > St. that is most likely the cause.  Smoke is billowing out of several
> > > manholes.  NGrid, NStar, police & fire are on the scene.  Lightower
> > > has dispatched techs.  No ETR yet.
> > >
> > > One other of my circuits that leaves the building via a diverse path
> > > is still up luckily, or we'd be off the net entirely.