A plumber with a blowtorch is the enemy of the data centre

On-Call Welcome again to On-Call, our regular week-ender in which readers share their tales of possibly-career-ending errors.

This week, reader “Harry” tells us he's done support for the same global company for the last thirteen years and in one role found himself doing network support.

One cold, cold winter's day, that gig took him into a pre-fabricated office, the kind used rapid-assembly housing or extra schoolrooms. At Harry's place of work this building had been designated as the “Safety Training Trailer.”

Harry was asked to visit the building because no data was getting in or out.

So he peered into the actual cupboard that served as the server and comms rack and found a switch merrily blinking away, with all ports lit up.

When Harry unplugged a cable, that port went off. When he plugged it back in, on came the light. But no data was moving.

After exhausting his diagnosis of the server, Harry wondered what was going on at Level 1 – the physical layer of the network. So he looked down and found a hole in the floor through which the switch's cables descended, noting just how unsafe that ragged excavation seemed.

Harry rugged up and went outside and traced the cables, which before long came close to the building's water pipes. Those conduits bore the unmistakable marks of recent blow-torch action. Which seemed like a pretty safe way to deal with the problem of frozen pipes.

As the data cables were next to the water pipes, they were a fused mess of copper. Another tick for safety!

“All of the copper wires were touching and causing loops on every port of the switch,” Harry recalls.

“Needless to say I got their manager out there, said there is your problem, and told them to have the idiots back out to re-run all of the cables,” he added. “Yes, the same people with the blowtorch were the same people that ran my cables.”

“Needless to say they put some better insulation on the pipe and used that as an example, in the safety training trailer.”

What's burned up your data centre? Write to let me know and you could end up in a future edition of On-Call. ®