Dude, relax – it's Just a Bunch Of Disks: Our man walks you through how JBODs work
Trevor Pott puts AIC's DAS subsystem through its paces
Two at the same time
Here's where the JBOD systems come in. SAS isn't just "SATA drives that can survive the loss of an HBA". SAS disks can be shared by two systems at the same time! JBOD units such as the AIC XJ3000-4243S I tinkered with are exactly how you do it.
Being external devices, JBODs use an external mini-SAS connector instead of an internal one. You get the relevant cable and plug your HBAs from two systems into the JBOD unit and you're on the road to a high-availability setup. To really get redundant you'd add two HBAs into each server and use two JBODs; that way you can lose either server or either JBOD and your storage cluster will continue.
You'll need an operating system that understands this, of course, but that doesn't look to be too big a problem. Windows's failover cluster manager makes this push-button simple in Server 2012 and *nix options certainly are available.
Among the *nix options available, Nexenta seems the most viable. They maintain their own Solaris fork with a laser focus on ZFS-based storage including exactly these sorts of JBOD configurations.
The device itself
The unit we tested is by no means the top-end offering. We only got to play with 24 disks in a single chassis, AIC can cram 60 disks into 4U with these things.
Even though we only got to play with the mid-range offering, I was not happy to see the XJ3000-4243S leave the lab. It has served well during its tenure, and there is something to be said for the ability to plug 24 disks into the thing and run disks tests against them all simultaneously. It cut about three weeks off our disk diagnostics and RMA backlog all on its own.
All the bits are there; JBOD setups as described above are a tried, proven solution set with enough maturity for there to be multiple players at every point in the market. The margins aren't outrageous and the $/GB of redundant, highly available storage can go up against the many object storage solutions that are floating around. How the hardware holds up would seem to be the determining factor.
My tests of the XJ3000-4243S showed that it was a fantastically built unit. It has hot-swappable power supplies that survived my rigorous "let's see if we can break it" testing. It's got hot-swappable fans that are easy to replace and take punishment. We tested a number of drives ranging from 5400 RPM SATA to 15,000 RPM SAS disks and it handled them all just fine.
It has a 1+1 600w redundant 80 Plus PSU, tool-less rail kit and the onboard firmware for the SAS expanders make it upgradable.
The XJ3000-4243S handled temperature extremes to 45C (impressive, as it's only rated to 35C) and had some of the least frustrating hot-swap trays I've ever used. If you want to chain up more JBODs, you can; it has its own external ports and so you can get a lot more disks in your cluster than just what's in the pair of JBODs.
Fitness for purpose
If your goal is to slam each and every disk to the absolute red line and eke out every conceivable IOPS from every spindle and NAND chip, this is not the solution for you. As discussed above, a JBOD attaches multiple hard drives to a far smaller number of SAS ports via the SAS expanders and the mini-SAS connecter to the server's HBA. You can use up all of the interface bandwidth fairly easily, depending on what you are doing to the disks.
Object-based storage systems can scale out to more physical disks than the JBOD high-availability setups that these devices are typically used to create, but there's nothing that says you have to buy JBODs to run them in HA setups. AIC and other JBOD chassis vendors offer models that are designed to maximise the number of disks per system as opposed to the focus on high availability.
There is nothing that – in theory – stops you from building object storage arrays where an individual "system" controls an entire rack's worth of disks. I can't find anyone doing that, however, largely because that is an awful lot of spindles to be made unavailable if a single server dies. Getting the balance just right is the focus of many a doctoral thesis and the fevered marketing of everyone from Supermicro to Nutanix.
You can now officially attach more disks to a single server than most sane people would ever want to. If your goal is to build a Massive Array of Idle Disks (MAID) then I can't see why you'd ever use anything else. Even if you are building some big, sexy storage backend that will eventually become a SAN for virtualisation, Nexenta's numbers on this argue that you can do quite well with this arrangement.
The next few years are going to contain the most bitterly fought battles the storage wars have seen for some time. Enterprise SAN vendors are under siege from all sides. JBOD HA approaches will sound off object-based storage, virtual storage appliances and network-backed block-level replication schemes. If you're a storage vendor then you are most certainly living in interesting times. ®