nanog mailing list archives
Mitigating human error in the SP
From: Chadwick Sorrell <mirotrem () gmail com>
Date: Mon, 1 Feb 2010 21:21:52 -0500
Hello NANOG, Long time listener, first time caller. A recent organizational change at my company has put someone in charge who is determined to make things perfect. We are a service provider, not an enterprise company, and our business is doing provisioning work during the day. We recently experienced an outage when an engineer, troubleshooting a failed turn-up, changed the ethertype on the wrong port losing both management and customer data on said device. This isn't a common occurrence, and the engineer in question has a pristine track record. This outage, of a high profile customer, triggered upper management to react by calling a meeting just days after. Put bluntly, we've been told "Human errors are unacceptable, and they will be completely eliminated. One is too many." I am asking the respectable NANOG engineers.... What measures have you taken to mitigate human mistakes? Have they been successful? Any other comments on the subject would be appreciated, we would like to come to our next meeting armed and dangerous. Thanks! Chad
Current thread:
- Mitigating human error in the SP Chadwick Sorrell (Feb 01)
- Re: Mitigating human error in the SP Suresh Ramasubramanian (Feb 01)
- Re: Mitigating human error in the SP Dobbins, Roland (Feb 01)
- RE: Mitigating human error in the SP Stefan Fouant (Feb 01)
- Re: Mitigating human error in the SP Ross Vandegrift (Feb 03)
- Re: Mitigating human error in the SP Christopher Morrow (Feb 03)
- Re: Mitigating human error in the SP Michael Dillon (Feb 03)
- Re: Mitigating human error in the SP David Hiers (Feb 03)
- Re: Mitigating human error in the SP Michael Dillon (Feb 02)
- Re: Mitigating human error in the SP Suresh Ramasubramanian (Feb 02)
- Re: Mitigating human error in the SP Steven Bellovin (Feb 02)
- Re: Mitigating human error in the SP Suresh Ramasubramanian (Feb 01)