Feeds

BOFH tests the law of redundant supply

P45s at the ready

  • alert
  • submit to reddit

HP ProLiant Gen8: Integrated lifecycle automation

Episode 33 BOFH 2004

Exciting times...

It seems the power supply of our webserver crapped itself and, in an excellent example of superior technology, grounded the phase for the rack - which in turn tripped the breaker and took out all the non-redundant powered systems within.

The PFY is on the job in a flash having seen exactly this type of thing many times before in kit from this vendor. (It's always reassuring to note that when a company merges with a company that in turn merged with the original hardware manufacturer, they still manage to incorporate the worst design features from the original product.)

But I digress.

"When's the website going to be up?" the Boss asks, sputniking around me madly.

"Not sure," I respond. "It'll depend on how long the engineer takes to get here."

"Can't you fix it?"

"I could, but it would void the warranty - me not being a certified engineer and all."

"Well, have you called it in?"

"Just getting round to it," I say, firing up the systems page on the browser to get the serial number. >scratchey< >scratch< “Righto, Sorted."

"Where are you going now?!" the Boss gasps as I get up.

"Just getting a coffee and some lunch."

"But it's only 11:30?!"

"Yeah but I am planning on waiting on the line till they answer..."

..later that day outside the Boss' office..

"So how important is it that we get the website back online?" I ask.

"Very!!" the boss blurts.

"Ah!"

"Why?"

"I told them it was a severity three call."

"That's good isn't it?"

"Not really. Watering their pot plants is Sev-2, so I doubt they'll be here today..."

"What?! Ring them back, change the severity!"

"Well, I could try calling, but it's 4pm now and their call desk closes at five."

"THAT ISN'T SERVICE!"

"No, but it's only a web server."

"Every minute the site's down we're losing money!" he gasps.

"No we're not. We don't do any business over the web so we're not losing money! Had it been the Stocks trading server at this time of day though, it'd be a completely different story and half the company would be banging on your door.."

"But the web's a valuable customer interface!"

"If you're Amazon, or Sendit, but not if you're us. We're a web nothing! Baby seals get more hits!"

Somewhere in the recesses of my mind I realise I may have stepped over a line here - if only in good taste.

"I..." the Boss says, slightly deterred, "... but it's core business!"

"Nah, that's just what the website company said. In reality, it could disappear without anyone noticing."

"I think that's an overstatement!"

"How much traffic does the website do on the average day?" I ask the PFY.

>clickety< >tap< >tap< >click<

"Hmm, about a meg. Including the index rebuild," The PFY responds.

"And the index rebuild generates?"

"Just under a meg."

"So in effect, the loss of our website would have as much impact..."

"As Karen Carpenter on a Salad Bar," the PFY says, joining me over this side of the line.

"Well I still want it up!" the Boss snaps, liking the feel of an executive decision.

"But the only way to get it up today would be to borrow the spare supply out of another server," the PFY suggests. "A more important server. We don't generally like to do it beca.."

"JUST DO IT!"

"Bu.."

"GET IT DONE!"

Five minutes and one large >clack< from the powerboard later.

"What the hell's happened?" the Boss blurts.

"The only similar power supply was in the Stocks and Tradings server, so I whipped that out to test if it would work, but the problem seems to be on the motherboard so it's cooked another supply," the PFY gabbles.

"YOU USED THE REDUNDANT SUPPLY FROM THE STOCKS TRADING SERVER!" the Boss gasps. "No."

"Oh thankgoodnessforthat!"

"No it seems it wasn't redundant after all."

"What are you telling me?" the Boss asks, paling slightly.

"You can take the power supply out and the machine will work ok."

"Yes?"

"..for about five minutes, then it overheats because the cooling fan in the power supply is missing from the machine," the PFY finishes.

"THAT'S BLOODY CRAP REDUNDANCY!" the Boss shouts.

"Sort of. You see, there's a fan unit somewhere that you remove when you install the redundant power supply."

"Then put it back in the machine!"

"There's the problem, no-one knows where it is, and there's less than a minute left."

"So what do we do?"

"I'm going to check the stock exchange website to see what the company's holding in its system and get a printout up to the traders. You might want to grab the newspaper."

"To check stock prices?"

"Nah. The job pages. I foresee another crap redundancy on the horizon..." ®;;;;;;;

Reducing security risks from open source software

More from The Register

next story
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
Amazon Reveals One Weird Trick: A Loss On Almost $20bn In Sales
Investors really hate it: Share price plunge as growth SLOWS in key AWS division
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
SHOCK and AWS: The fall of Amazon's deflationary cloud
Just as Jeff Bezos did to books and CDs, Amazon's rivals are now doing to it
BlackBerry: Toss the server, mate... BES is in the CLOUD now
BlackBerry Enterprise Services takes aim at SMEs - but there's a catch
The triumph of VVOL: Everyone's jumping into bed with VMware
'Bandwagon'? Yes, we're on it and so what, say big dogs
Carbon tax repeal won't see data centre operators cut prices
Rackspace says electricity isn't a major cost, Equinix promises 'no levy'
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Seven Steps to Software Security
Seven practical steps you can begin to take today to secure your applications and prevent the damages a successful cyber-attack can cause.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.