nanog mailing list archives

RE: Famous operational issues


From: "Tony Wicks" <tony () wicks co nz>
Date: Tue, 23 Feb 2021 09:42:55 +1300

Many years ago I experienced a very similar thing. The DC/Integrator I worked for outsourced the co-location and 
operation of mainframe services for several banks and government organisations. One of these banks had a significant 
investment in AS/400's and they decided that it was so much hassle and expense using our datacentres that they would 
start putting those nice small AS/400's in computer rooms in their office buildings instead. One particular computer 
room contained large line printers that the developers would use to print out whatever it is such people print out. One 
Saturday morning I received a frantic call from the customer to say that all their primary production as/400's had gone 
offline. After a short investigation I realised that all the offline devices wire in this particular computer room. It 
turn's out that one of the developers had bought his six year old son to work that Saturday and upon retrieval of a 
printout said son had dutifully followed dad in to the computer room and was unable to resist the big red button 
sitting exposed on the wall by the door. Shortly thereafter the embarrassed customer decided that perhaps it was worth 
relocating their as/400's to our expensive datacentres.




     During my younger days, that button was used a few time by the 
operator of a VM/370 to regain control from someone with a "curious 
mind" *cought* *cought*...

Two horror stories I remember from long ago when I was a console jockey for a federal space agency that will remain 
nameless :P

1. A coworker brought her daughter to work with her on a Saturday overtime shift because she couldn't get a babysitter. 
She parked the kid with a coloring book and a pile of crayons at the only table in the console room with some space, 
right next to the master console for our 3081. I asked her to make sure sh was well away from the console, and as she 
reached over to scoot the girl and her coloring books further away she slipped, and reached out to steady herself. Yep, 
planted her finger right down on the IML button (plexi covers? We don' need no STEENKIN' 
plexi covers!). MVS and VM vanished, two dozen tape drives rewound and several hours' worth of data merge jobs went 
blooey.



Current thread: