Re: Centurylink having a bad morning?

[Joseph Jenkins <joe () breathe-underwater com>]

Latest updates from my tickets:
08/30/2020 14:28:20 GMT - The IP NOC confirmed a routing issue and
commenced with troubleshooting efforts. Routing configuration adjustments
have been made and service affecting alarms are beginning to clear.

08/30/2020 11:38:15 GMT - The IP NOC is engaged in cooperative escalated
investigations to isolate and troubleshoot the fault at this time.

08/30/2020 11:03:09 GMT - On August 30, 2020 at 10:00 GMT, CenturyLink
identified a Market Wide service impact. As this network fault is impacting
multiple clients, the event has increased visibility with CenturyLink
leadership. As such, client trouble tickets associated to this fault have
been automatically escalated to higher priority.

The NOC is engaged and investigating in order to isolate the cause. Please
be advised that updates for this event will be relayed at a minimum of
hourly unless otherwise noted. The information conveyed hereafter is
associated to live troubleshooting effort and as the discovery process
evolves through to service resolution, ticket closure, or post incident
review, details may evolve.



On Sun, Aug 30, 2020 at 7:30 AM Antonios Chariton <daknob.mac () gmail com>
wrote:

> Reporting from Europe, any IP with them in the path is unreachable from
> various providers. I guess they wanted to try IPv6-only.. :P IPv6 is fine,
> working fine, IPv4 not at all..
>
> Antonis
>
> > On 30 Aug 2020, at 14:58, Tomas Lynch <tomas.lynch () gmail com> wrote:
> >
> > Flapping in Miami, Dallas, Atlanta, Los Angeles, Seattle and San Jose.
> It is also affecting some data centers in Europe too. but haven't seen
> flaps there, just suboptimal routing.
> > On Sun, Aug 30, 2020 at 8:53 AM Drew Weaver <drew.weaver () thenap com>
> wrote:
> > Saw the flapping in Cleveland but not in Cincinnatti or Ashburn…
> >
> >
> >
> > From: Tomas Lynch <tomas.lynch () gmail com>
> > Sent: Sunday, August 30, 2020 8:45 AM
> > To: Mel Beckman <mel () beckman org>
> > Cc: Drew Weaver <drew.weaver () thenap com>; nanog () nanog org
> > Subject: Re: Centurylink having a bad morning?
> >
> >
> >
> > BGP sessions randomly flapping or having routing issues in different
> cities since ~5AM EST
> > On Sun, Aug 30, 2020 at 8:42 AM Mel Beckman <mel () beckman org> wrote:
> >
> > The CL portal loads for me, and I can log in, but it is slower than
> usual. Not seeing traffic issues on our CL circuits.
> > -mel via cell
> >
> >
> >
> >
> > On Aug 30, 2020, at 5:23 AM, Drew Weaver via NANOG <nanog () nanog org>
> wrote:
> > 
> >
> > Hello,
> >
> >
> >
> > Woke up this morning to a bunch of reports of issues with connectivity
> had to shut down some Level3/CTL connections to get it to return to normal.
> > As of right now their support portal won’t load:
> https://www.centurylink.com/business/login/
> > Just wondering what others are seeing.