Waiter? There's a mouse in my motherboard and this server is greasy!
Support chap reveals the squalid horror of restaurant computers
ON-CALL Welcome again to On-Call, our weekly sharing session in which readers unburden themselves by sharing memories of nasty jobs.
This week meet “Charlie” who wrote to tell us he used to have a gig maintaining point of sale systems for a a restaurant chain. Charlie wrote to tell us about the day “I was called to one of the units to investigate why the 'main' computer was running but none of the terminals had any communication.”
“These were green-screen or amber-screen dumb terminals, printers, and various other devices running off a terminal server board over 38,400 cps communication hosted by a Pentium I computer running Red Hat or SCO Unix and some proprietary software,” Charlie told On-Call.
“Most of these were NCR-branded computers” Charlie says, noting that “NCR used to build a hell of a reliable machine.” But no machine will thrive in the kind of environment he often found on site.
“Some of these machines were 10 or more years old and had practically zero preventative maintenance due to the parent company being perpetual tightwads,” he told The Register. “Most were fairly dirty, some filthy. All fans were usually dead and coated with a thick layer of sticky goo from the grease in the air in the restaurants.”
The machines ran 24/7 and could clearly stand the heat of a kitchen, “possibly owing to the massive aluminium passive heatsinks on the processor and other critical chips.”
So when Charlie showed up at the restaurant in question he was expecting only familiar horrors as the source of the on-screen error “Message from syslog: Server timeout waiting for port xxx to respond.”
Because these machines often got jostled and cables fell out, Charlie checked the various connections and found all present and correct. He rebooted it, but had no luck.
But he did begin to think that the box connecting all the POS terminals to the server might be a problem because none of its lights were on. That said box was connected to the server with a thick SCSI cable led him to think a comms card had failed.
There was therefore nothing for it but to open the server. So Charlie “pulled the cover off this filthy machine and was immediately met with an awful stench and thousands of mouse turds and some liquid, plus scraps of shredded paper, lint, and fur.”
“Clearly a mouse and his or her family had gotten into the case through a missing blank plate in the back and had made the nice warm case their cozy home for some time.” And they'd put that time to good use “peeing and crapping on everything in the case until finally the urine corroded through a trace on the riser card that fed the PCI cards. (the copper traces were green with a white powdery substance) Despite all this, the machine was still running, albeit with no throughput on the PCI bus.”
Charlie wrote the thing off immediately, pulling out its disks before “the rest of the whole reeking biohazard mess went into the dumpster, at arm's length.”
“I then proceeded to wash my hands for about five minutes before configuring the replacement I'd brought.”
And if you think that's gross, know that Charlie says the mouse was never found. But its traces were: he says it produced “a lingering redolence in the back of the restaurant for weeks to come for the kitchen workers to enjoy.”
If you've found a computer in worse condition that Charlie's restaurant horror, write to share your story and it could be your anonymised name filling the On-Call slot next week. ®
Sponsored: Becoming a Pragmatic Security Leader