Developer mistakenly deleted data - so thoroughly nobody could pin it on him!
Don't ask your staff to write scripts at beer O'clock on Friday afternoon
Who, me? Welcome to the eighth edition of "Who, me?", the column in which Reg readers confess to moments at which they messed things up but good.
This week meet "Ben" who told us that "On a Friday afternoon about five years ago I was asked to complete some backup scripts before the weekend."
Ben told us his employer "was too tight to pay for any software to manage this and so a flurry of bash scripts were developed in haste." The company was, however, content to have hastily-written scripts protect document repositories, databases, financial data and Apache Subversion files.
Nothing important then. Just the critical stuff!
Ben wrote the scripts, went home and thought nothing of it until Monday morning when he arrived at work and found "certain shares weren't available."
"I trundled off to the server room, a small air-conditioned room with a single cabinet, switched on the monitor and hit a key," Ben told us in his email to Who, me? "The openSUSE Enterprise desktop came up, which was encouraging. "But it was unresponsive," which was discouraging.
Sysadmin left finger on power button for an hour to avert SAP outageREAD MORE
"A little investigation led to the reset of the machine which hung at the bootloader. Using a live boot CD I booted the machine and discovered that the hard disks in the machine were, well, blank - except for a few empty directories in the root directory."
Ben's not entirely certain what happened, but said he "assumed one of my scripts, developed in haste, on that Friday afternoon had done the unthinkable... It must have executed a 'rm -rf //' - where between the slashes should have been a temporary folder to remove. In some form of irony, the backup scripts were also deleted in this process - so all data, and all evidence, was gone."
"I immediately handed the problem over to one of the senior engineers (I was just software developer, who looked after the I.T as well) explained that I can only assume my backup scripts had wiped the server."
"He removed the disks, sent them off for recovery - which incidentally failed to recover any data, and set me about rebuilding from manual backups taken some three months ago."
"I was never given a grilling. To be fair, I gave myself a fairly hard time, but without evidence and only assumptions maybe it wasn't me."
Have you messed something up and escaped censure? If so, click here to send us your story and it might pop up in a future edition Who, me? ®