Sysadmin unplugged wrong server, ran away, hoped nobody noticed
‘I was a snot-nosed kid fresh out of college and thought I knew everything!’
Who, me? Another working week beckons so once again let’s kick it off with a fresh instalment of Who, me? For those of you new to the column, it's The Register confessional for IT pros who broke things.
This week we meet “Hayden”, who confessed that: “Back in 1992 I was sort of a snot nose kid just a few years out of college that knew everything!”
Everything except how to follow a power cord to a socket.
Hayden discovered his lack of skills in this regard when working for a telecoms research business that did mostly research, but also wrote production software for the regional telcos in the US.
The research focus meant the company’s server room was a little shambolic.
'I crashed AOL for 19 hours and messed up global email for a week'READ MORE
“There were many Sun SPARC servers in this room and when a research group got a new one, they just wheeled it in and put it where ever there was floor space, pulled up a floor tile and found a free power socket and Ethernet plug.”
This arrangement meant that one server room housed development, test, and production servers. Hayden told us none of that mattered – the team just did the job.
And so did he, which one day meant moving a test server from the server room to his office.
“It was having some hard drive issues and constantly having to run back and forth the two flights of stairs to the server room just wasn’t making sense, so my brilliant idea was to unplug it and take it to my office!:
So Hayden pulled up the floor tiles and felt around to find where his server’s power cable terminated.
“Cables were just in piles, no zip ties, no cable tracks, just four inches of open space between the floor tiles and the concrete floor, most of that occupied by various cables.”
“I did my best to trace the power cable from my test server to an outlet,” Hayden told us. But after a few frustrating minutes, “I decided the easy approach was just tug on the cable at the point I knew it was my power cable and with another hand wait for a tug at each of the plugs in the outlet.”
“After five or six tugs the plug moved. I was like ‘bingo’, and promptly pulled the plug.”
Hayden expected to see his server power down, but its blinkenlights kept blinking which was unexpected. As was the sound of one of the larger servers’ fans winding down. And when Hayden looked at that beast its blinkenlights most definitely were not blinking.
“So I did what any 23 year-old syadmin that knew everything would do,” Hayden said. “I put the plug back in, dropped the floor tile and walked as fast as I could to my office!”
And he got away with it too: Hayden said he returned to the server room about 40 minutes later to join a small group trying to figure out why their server lost power and discussing how long it would take for all of its X.25 connections to recycle. As said server had about 20 or so connections, it was four hours before it resumed duty.
And of course this was one of the few production servers in the room, meaning a bunch of telcos weren’t able to place circuit orders for a while.
Hayden told us that he tried again to retrieve his server a few days later.
“This time I pulled more floor tiles so I could trace the power cord properly. Close to the power plug, the two power cords literally wrapped themselves around each other so tight, that yeah, when I pull the correct cord, both cords would come out of the plug.”
Have you managed to evade responsibility for a big boo-boo? If so, click here to write to Who, me? and we’ll try to give you your moment in the sun on some future Monday. ®