Thundering gas destroys disks during data centre incident
Keep eating the curries, sysadmins, but consider a new nozzle for gaseous emissions
A few weeks ago in a Sydney data centre, something started making smoke. As its designers intended, the centre's reservoirs of inert gas started venting to stop any fire spreading.
The smoke was contained, but not long afterwards tenants started to complain that some of their disks had not survived the incident, which was odd. Reports of the incident The Reg encountered suggest that not much smoke was produced and that servers and other kit did not suffer. One tenant reported that SAS disks fared worse than newer SATA kit and drives in arrays did better than those in servers.
Vulture South asked around among disk and array vendors to learn if they are aware of anything that might have caused the problem. None could think of a single thing, which put paid to theories that perhaps the hypoxic gases used in data centres might be inimical to drives innards.
We also asked data centre construction specialists, one of whom – Emerson Network Power's Per Grandjean-Thomsen – recalled a 2012 Siemens white paper that suggested the noise made by rapidly-escaping hypoxic gas was the culprit. Extra noise, the paper suggests, means more vibration than some disks can handle. That white paper turns out to have a 2009 progenitor (PDF) explaining tests that are the source of the gas-release-noise-destroys-disks theory.
The noisy gas hypothesis stacks up with observations from Sydney. SAS disks are not hard to come by in lower volumes than their SATA cousins, which probably means less innovation. It's also to be expected that arrays tuned to handle dozens or hundreds of disks offer more vibration-damping features than servers.
The good news is that Siemens' research saw it create a gas-release nozzle that spreads inert gas inside a data centre without making such a racket. The data centre where this incident took place is one of Sydney's older such facilities, which could explain why it still uses noisy nozzles.
Has the data centre where your kit lives made that upgrade? Perhaps the time has come to make some noise about it. ®
Bootnote For another look at how vibration impacts disks, devote two minutes of your life to this classic vid "Shouting in the data centre", from back when Sun made its own hardware.