You shrunk the database into a
.gz and the app won't work? Sigh
Plus: The misguided tale of the 20-second support response time that wasn't
On-Call Welcome yet again to On-Call, our unimaginatively-named regular recap of readers' recollections of jobs gone wrong.
You all liked a double-barrelled story so much last week we've decided to do it again, starting with “Gary” wrote to tell us of his time working for Burroughs and the job he was sent to do for a bank with a branch on Otley Road in the British city of Bradford.
Gary mis-heard the address and instead went to the bank's branch in the town of Otley, about 15 kilometres away and a half-hour's drive when the traffic is kind.
Gary walked into the Otley branch just in time to see the local tech putting the phone down and expressing amazement at Burroughs 20-second response time.
Meanwhile at the Otley Road branch, the customer was rather less impressed!
Also unimpressed is reader “Brad” who wrote to tell us of the time he worked for the UK Ministry of Justice where he helped writing a system to administer prisons.
While Brad and his mate developed the software, administration and ops were outsourced.
To hold up his end of the bargain and do things like debugging, Brad often needed a snapshot of the production system.
On one such occasion he asked the outsourced DBAs to make a backup of the Oracle database and send a copy over.
As Brad explains “This was easily done at night by stopping the application and running a database export to obtain a consistent set of data.”
Yup – nothing scary there. So why did Brad get a call the next morning asking why the application wasn't running? He couldn't imagine why: this kind of data dump was and is routine.
Brad investigated and it turned out the usual DBA was on holiday and the fill-in “had gone into the directory hosting the database files and were going to copy them to a tape but noticed they were too large to fit.”
“So what did they do next? They ran
gzip on their running production system and compressed the files in-situ.”
Yes. They compressed the live production database. With something other than the database's native encryption too.
“Needless to say,” Brad says, “we uncovered why the application was no longer operational!
What mistakes has an outsourcer dumped in your lap? Click here to revive the lost art of letter-writing and let me know your story. Who knows? You could end up in this very slot on some future Friday. ®
Sponsored: Beyond the Data Frontier