Feeds

BOFH and The Engineer

Call out beyond sense of duty

  • alert
  • submit to reddit

Designing a Defense for Mobile Applications

Episode 13

BOFH 2003: Episode 13

BOFH logo Picture if you will a computer room, late at night..

Late, late at night...

Almost morning, in fact....

All lights - bar several system status indication lamps - are off.....

Suddenly a lamp, previously glowing Blue, changes (hereafter) to Orange.......

Seconds later an SNMP trap, delivered by the system concerned is received by the Services Monitoring Facility server...........

Seconds after that, a dial tone, followed by a series of DTMF tones is heard echoing through the machine room. And again...............

Approximately one minute later, in the bedroom of a geekily furnished studio flat in South London, a pager beeps shrilly .................................................

A figure partially emerges from the Fleur-de-Lis bedspread to focus on the pager in question........................................

THE WELL-OILED MACHINE OF TRIPLE-TIME, AFTER-HOURS CALLOUT IS IN MOTION!!!!

. . .

"But I didn't get paged!" The PFY cries in response to The Boss's annoyed questioning as I roll in at 9 the same morning. "Mind you, that could have been because I dropped the pager on the way home and it broke!"

The PFY holds up a pager that, whilst bearing the hallmarks of being repeatedly struck by the heel of a shoe, must have been damaged in the manner he said...

"I did mention the need of some form of backup to the pager in case of situations like this - or pager unreliability due to congested telco networks," I ad lib.

"I'm not buying a bloody satellite phone for you to run up astronomical bills on!" The Boss snaps, cutting my plan off at the pass. "Anyway, this isn't getting the Finance Server up!"

"The Finance Server IS up!" I say, looking on the Services Monitor. "It's just dropped a power supply"

"Then how come I got messaged?" he snaps, flashing us a look at the message on his phone.

"You said you must be notified of any major problems - and you know how I don't like to ask questions."

"That's not major!" he snaps.

"It is if it happens twice!" I reason.

"Look, I only want to be notified when the machine goes down!"

"OK, that's easily sorted out," I respond helpfully.

"And get that power supply fixed!" He snaps.

"Running all the way to the phone!" The PFY blurts, poking in the Hardware Support number.

Three hours later

"I thought I told you I only want to be paged when the server goes down!" the Boss seethes, crashing into Mission Control.

"Yes?"

"Well?"

"Oh! The server's gone down, see?" I say, pointing at a red icon on the services display.

"You're joking!" The Boss blurts. "But it's got redundant power supplies!"

"Indeed it has, but they don't work when one's dead and the other is outside the machine..."

"I thought the machines could work without interruption?"

"They can, but unfortunately our engineer can't. The PFY asked him if he wanted a coffee, which must have confused 'LEFT' and 'RIGHT' in his brain. So he pulled the wrong supply out. I was going to check before he did it - but as you know, I don't like to ask questions..."

"RIGHT!" The Boss shouts as he storms off to his office.

Ten minutes later The Boss informs us that the engineer concerned is on his way back to the office and the senior engineer has been dispatched to us.

...

"..a real bastard," I say to The PFY, finishing my description of the engineer in question. "The sort of person who gives helpful tips to your boss like how the dangling cables you have are a workplace hazard and how he couldn't help noticing that you're using root for day-to-day work - and how insecure that is.."

"Really?"

"Oh yes. An out and out brownnoser. A cleft presser of the First Order. AND he notes all his tips and observations on his report and files it with his office. THEN, next time something happens, they dig through it for a reason to void the maintenance agreement."

"Really?"

"Oh yes. He's the guy they send to punish people. AND, he makes a point of coming in and advising you through what he's doing, every step of the way."

"Arse Covering?" the PFY asks.

"Partly, but mainly to punish you for not issuing him a permanent swipe card to access the room night and day. So he wanders in and out of the machine room to make you keep getting up to let him in."

Our conversation is interrupted by the arrival of the man himself, complete with titanium toolkit.

"So where is it?" he asks.

"In the machine room," I respond.

"OK, can you give me a card to get in and out?"

And so it begins...

Ten minutes later...

"Just about to install the new power supply," he says, once the computer room door has latched shut.

"Good."

"And could someone let me into the machine room, as I don't have a c.."

"Sure," The PFY says cheerfully

Ten minutes later...

"New power supply installed, I'm just going to box up the old one >click< and fill in my form."

"Uh-huh".

"Oh, could you let me back into the..."

"Sure," The PFY seethes.

. . .

Looking back, I think it was probably the trip back to collect his toolkit that broke The PFY. The repeated trips in and out to let us know he was about to power the system on, check the hot plugability was working, then confirming it was all working, boxing up the old power supply, borrowing a pen to label the old power supply 'faulty', packing up to go, etc, must have slowly eroded his patience.

The phonecall from the Maintenance Company asking if we'd seen their engineer was the first thing I knew about it.

Then there was The PFY's new titanium tool kit.

I don't like to ask questions... ®

BOFH is copyright © 1995-2003, Simon Travaglia. Don't mess with his rights.

The Power of One eBook: Top reasons to choose HP BladeSystem

More from The Register

next story
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
Attack of the clones: Oracle's latest Red Hat Linux lookalike arrives
Oracle's Linux boss says Larry's Linux isn't just for Oracle apps anymore
THUD! WD plonks down SIX TERABYTE 'consumer NAS' fatboy
Now that's a LOT of porn or pirated movies. Or, you know, other consumer stuff
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.