BOFH: Woah there, Crash Gordon
Episode 32 "WOAH THERE, Crash Gordon!" the PFY shouts as our engineer prepares to put our system back together "How's about you go get another disk drive!?"
"Huh?" the engineer asks blankly. "I just replaced it."
"Yeah, then you dropped the drive and the tray off the desk onto the floor!"
"It won't harm the disk, they're rated to 9 Gs!"
"Save it for some who believes you!" the PFY snaps disinterestedly.
"But it's a valid replacement disk - and it's new, not a service drive!"
"I don't give a crap if it's new, it's been dropped!"
"But it can take it!"
"If it was designed to 'take it' they wouldn't pack it in foam AND WRITE FRAGILE ON THE BLOODY BOX!"
"It's probably an old box," he pinocchios.
"No, it's a new box," the PFY says, fingering some barcode gibberish on the side which probably means something to people without lives.
"Well let's just see if it fires up, eh?" he asks.
"DON..." the PFY shouts as the engineer flips the power switch.
. . .
"There, see, it's seen the drive!" >Clickety< he says, firing up the SCSI card BIOS tools. "Hmmm, Low level format with error checking just to be sure it's ok... >click< uh... ...YES I'm sure.."
>click< "Too, late, it's running! The quicker we get it formatted the quicker it'll be in service!"
"Uh," the PFY says, suppressing anger. "You've just started a low level format of the surviving member of the mirrored set - not the disk you just replaced"
"I... Oh. Well, it's a RAID set, so you'll be able to recover it from the other drive."
"... Tell me, did you actually do any computing training when you changed careers from rounding up stock?" the PFY seethes.
"He's implying that you're a complete cowboy who doesn't know the first thing about computing because if you did you'd know you'd just destroyed our dataset," I say helpfully, reaching for the computer room phone and tapping out the service number.
"Don't you do backups then?" he asks.
. . .
Two hours later he's gone and a replacement's arrived.
"So what's the problem again?"
"He removed the failed disk, put a new one in the hot plug tray, dropped the tray on the ground, then put it into the machine then formatted the wrong disk," I say.
"Oh," he says. "So you've lost your data. But surely you'll just recover it from backup? I mean there's not a lot I can do."
"We'd like you to replace the disk he dropped," I say, kindly.
"?! It's working isn't it?"
"It's working now," I say slowly. "But its lifetime is likely to be severely shortened by the mistreatment, which means it'll probably fail in a couple of months - or hours - instead of a couple of years."
"Well firstly the company would want proof that our engineer dropped the drive - which I'm guessing you don't have, but secondly the drive's actually working so I can't replace it - there's nothing wrong with it!.. uh... Sorry."
"Ok, lets put it another way. You can order a disk now, have it couriered here and spend a couple of hours lazing around having coffee and biscuits, or you can leave now and have to come back in a couple of hours to replace the failed disk."
"It might not fail within a couple of hours!"
"Trust me - it will!"
"You'll void your warranty if you damage the disk!" he blurts.
"As if we would! You know there were times when we'd run an aggressive disk exercise pattern on the disk until it crapped itself, but these days we're much less sneaky. These days we just use a rubber mallet."
"...On the engineer until he replaces the disk..." the PFY adds.
"It'll leave marks!" he says.
"On you or the disk?" I ask.
"On the disk. There's a misuse indicator inside the drive, they'll know as soon as they open the drive!" he gasps.
"And what would this indicator look like after the drive's been dropped?"
"I... ... I still can't replace the disk - it's all inventory controlled. They know it was working"
"But they know it could fail at any time..."
"Any disk could fail at any time - it could be the one in a million that fails the moment it's installed."
"So why don't we say that it failed just now."
"Because they'll do the diags on it back at base and find out it's still working, and then I'll get it in the neck for not diagnosing it properly."
"So what you're saying is you'd like to help us but the drive needs to be dead before you take it back."
"Yes!" he gushes happily, having made his point.
"And you'd really like to help us out?"
Two minutes later, as the PFY and I are watching the engineer pounding the side of the hard drive with a rubber mallet, I can't help thinking that everything's going to be all right after all.
"No one will ever believe you talked him into that," the PFY says, shaking his head.
"No, that's why I'm taping it."
"Smooth!" the PFY nods appreciatively, reaching for a coffee.
"No, smooth is the fact that he's currently beating to death a drive from another machine which isn't even under warranty... On tape..." ®
Sponsored: Becoming a Pragmatic Security Leader