nanog mailing list archives
Re: Fiber cut in SF area
From: George William Herbert <gherbert () retro com>
Date: Mon, 13 Apr 2009 17:32:49 -0700
Matthew Petach writes:
"protected rings" are a technology of the past. Don't count on your vendor to provide "redundancy" for you. Get two unprotected runs for half the cost each, from two different providers, and verify the path separation and diversity yourself with GIS data from the two providers; handle the failover yourself. That way, you *know* what your risks and potential impact scenarios are. It adds a bit of initial planning overhead, but in the long run, it generally costs a similar amount for two unprotected runs as it does to get a protected run, and you can plan your survival scenarios *much* better, including surviving things like one provider going under, work stoppages at one provider, etc.
This completely ignores the grooming problem. About five years ago, we had a major WebEx outage caused by our diverse path routed fibers both being groomed into the same new cable / new path. We had the contracts. We paid the money. We got the data. We got updates to the data. The updates said we were still fine and all good. The new data lied. Downtown SJ backhoe hit damaged the cable, and took down 1 of our 2 links. As nobody was sure what was in it they failed to notify us that they were about to chop the rest of it to repair the bundle. So, about an hour after we lost the first leg, we went dark, and there was no coming back until the splices were all done. (typically, while the whole operations team was out at an offisite teambuilding effort. pagers go beep beep beep, and everyone hops back in the cars...) We ran it up the flagpole to CEO level of the fiber vendor (aggregator) and fiber physical plant owner (big 4 ISP), as we were paying $$$ for bandwidth and were a Highly Visible Client, and were told that they'd been making a best effort and couldn't guarantee any better in the future, no matter how much we paid or who we sued. They were very apologetic, but insisted that best effort means just that. The only way to be sure? Own your own fiber. Use a microwave link backup. You have to get out of the game the fiber owners are playing. They can't even keep score for themselves, much less accurately for the rest of us. If you count on them playing fair or right, they're going to break your heart and your business. -george william herbert gherbert () retro com
Current thread:
- Re: Fiber cut in SF area, (continued)
- Re: Fiber cut in SF area Mike Lewinski (Apr 11)
- Re: Fiber cut in SF area Joe Greco (Apr 12)
- RE: Fiber cut in SF area Dylan Ebner (Apr 13)
- RE: Fiber cut in SF area Mikael Abrahamsson (Apr 13)
- Re: Fiber cut in SF area Andy Ringsmuth (Apr 13)
- Re: Fiber cut in SF area Dorn Hetzel (Apr 13)
- Re: Fiber cut in SF area Dorn Hetzel (Apr 13)
- Re: Fiber cut in SF area Justin M. Streiner (Apr 13)
- Re: Fiber cut in SF area joel . mercado (Apr 13)
- Re: Fiber cut in SF area Matthew Petach (Apr 13)
- Re: Fiber cut in SF area George William Herbert (Apr 13)
- Re: Fiber cut in SF area Matthew Petach (Apr 13)
- Re: Fiber cut in SF area George William Herbert (Apr 13)
- Re: Fiber cut in SF area Matthew Petach (Apr 13)
- Re: Fiber cut in SF area Christopher Hart (Apr 13)
- Re: Fiber cut in SF area Jorge Amodio (Apr 14)
- RE: Fiber cut in SF area Murphy, Jay, DOH (Apr 14)
- Re: Fiber cut in SF area Jorge Amodio (Apr 14)
- RE: Fiber cut in SF area Murphy, Jay, DOH (Apr 14)
- RE: Fiber cut in SF area Peter Beckman (Apr 13)
- RE: Fiber cut in SF area chris.ranch (Apr 13)