Feeds

BOFH tests the law of redundant supply

P45s at the ready

  • alert
  • submit to reddit

High performance access to file storage

Episode 33 BOFH 2004

Exciting times...

It seems the power supply of our webserver crapped itself and, in an excellent example of superior technology, grounded the phase for the rack - which in turn tripped the breaker and took out all the non-redundant powered systems within.

The PFY is on the job in a flash having seen exactly this type of thing many times before in kit from this vendor. (It's always reassuring to note that when a company merges with a company that in turn merged with the original hardware manufacturer, they still manage to incorporate the worst design features from the original product.)

But I digress.

"When's the website going to be up?" the Boss asks, sputniking around me madly.

"Not sure," I respond. "It'll depend on how long the engineer takes to get here."

"Can't you fix it?"

"I could, but it would void the warranty - me not being a certified engineer and all."

"Well, have you called it in?"

"Just getting round to it," I say, firing up the systems page on the browser to get the serial number. >scratchey< >scratch< “Righto, Sorted."

"Where are you going now?!" the Boss gasps as I get up.

"Just getting a coffee and some lunch."

"But it's only 11:30?!"

"Yeah but I am planning on waiting on the line till they answer..."

..later that day outside the Boss' office..

"So how important is it that we get the website back online?" I ask.

"Very!!" the boss blurts.

"Ah!"

"Why?"

"I told them it was a severity three call."

"That's good isn't it?"

"Not really. Watering their pot plants is Sev-2, so I doubt they'll be here today..."

"What?! Ring them back, change the severity!"

"Well, I could try calling, but it's 4pm now and their call desk closes at five."

"THAT ISN'T SERVICE!"

"No, but it's only a web server."

"Every minute the site's down we're losing money!" he gasps.

"No we're not. We don't do any business over the web so we're not losing money! Had it been the Stocks trading server at this time of day though, it'd be a completely different story and half the company would be banging on your door.."

"But the web's a valuable customer interface!"

"If you're Amazon, or Sendit, but not if you're us. We're a web nothing! Baby seals get more hits!"

Somewhere in the recesses of my mind I realise I may have stepped over a line here - if only in good taste.

"I..." the Boss says, slightly deterred, "... but it's core business!"

"Nah, that's just what the website company said. In reality, it could disappear without anyone noticing."

"I think that's an overstatement!"

"How much traffic does the website do on the average day?" I ask the PFY.

>clickety< >tap< >tap< >click<

"Hmm, about a meg. Including the index rebuild," The PFY responds.

"And the index rebuild generates?"

"Just under a meg."

"So in effect, the loss of our website would have as much impact..."

"As Karen Carpenter on a Salad Bar," the PFY says, joining me over this side of the line.

"Well I still want it up!" the Boss snaps, liking the feel of an executive decision.

"But the only way to get it up today would be to borrow the spare supply out of another server," the PFY suggests. "A more important server. We don't generally like to do it beca.."

"JUST DO IT!"

"Bu.."

"GET IT DONE!"

Five minutes and one large >clack< from the powerboard later.

"What the hell's happened?" the Boss blurts.

"The only similar power supply was in the Stocks and Tradings server, so I whipped that out to test if it would work, but the problem seems to be on the motherboard so it's cooked another supply," the PFY gabbles.

"YOU USED THE REDUNDANT SUPPLY FROM THE STOCKS TRADING SERVER!" the Boss gasps. "No."

"Oh thankgoodnessforthat!"

"No it seems it wasn't redundant after all."

"What are you telling me?" the Boss asks, paling slightly.

"You can take the power supply out and the machine will work ok."

"Yes?"

"..for about five minutes, then it overheats because the cooling fan in the power supply is missing from the machine," the PFY finishes.

"THAT'S BLOODY CRAP REDUNDANCY!" the Boss shouts.

"Sort of. You see, there's a fan unit somewhere that you remove when you install the redundant power supply."

"Then put it back in the machine!"

"There's the problem, no-one knows where it is, and there's less than a minute left."

"So what do we do?"

"I'm going to check the stock exchange website to see what the company's holding in its system and get a printout up to the traders. You might want to grab the newspaper."

"To check stock prices?"

"Nah. The job pages. I foresee another crap redundancy on the horizon..." ®;;;;;;;

High performance access to file storage

More from The Register

next story
Seagate brings out 6TB HDD, did not need NO STEENKIN' SHINGLES
Or helium filling either, according to reports
European Court of Justice rips up Data Retention Directive
Rules 'interfering' measure to be 'invalid'
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
Bored with trading oil and gold? Why not flog some CLOUD servers?
Chicago Mercantile Exchange plans cloud spot exchange
Just what could be inside Dropbox's new 'Home For Life'?
Biz apps, messaging, photos, email, more storage – sorry, did you think there would be cake?
IT bods: How long does it take YOU to train up on new tech?
I'll leave my arrays to do the hard work, if you don't mind
prev story

Whitepapers

Mainstay ROI - Does application security pay?
In this whitepaper learn how you and your enterprise might benefit from better software security.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mobile application security study
Download this report to see the alarming realities regarding the sheer number of applications vulnerable to attack, as well as the most common and easily addressable vulnerability errors.