Got it taped: The business of tape-based disaster recovery
Taking a risky - or risk free? - hike up Iron Mountain
The other side of the coin works in Iron Mountain’s favour: people don’t like to delete anything, which surely means the shelves in the vault are groaning with more data than ever before? Yet Lloyd has witnessed some interesting changes to how the vault operates.
“We have noticed how the bigger capacity tapes effectively shrink what we store, but there’s more data arriving now than was seen five to ten years ago.”
So does this mean the place is getting stuffed with LTO-6 tapes now – the 2.5TB format released at the end of last year? Apparently not, most of the tape cartridges are LTO-4, with new system builds featuring LTO-5. Glancing at the shelves, it seems that Iron Mountain is no stranger to a regular, but small scale turnover of DAT tapes too.
Hal has yet to see LTO-5 take off with his clients and he doubts many have heard of LTO-6.
“We handle almost entirely LTO-4 tapes, a few LTO-3 from a legacy system. We’ve stuck with LTO-4 because you can get 1.6TB on each tape [using 2:1 compression] and they cost about £20 each. The price has come down a lot. They used to be £50 about five years ago.”
All in hand: tape storage at Iron Mountain
The purchasing and turnover of tape works out as a very modest investment. Admittedly, the tape library machines from the likes of Quantum, IBM and HP are expensive, but in Hal’s experience, for the most part, this equipment is extremely reliable. The tapes are too, but they do have their moments.
“The tapes are so resilient that unless they are physically damaged they are good for many, many years. Very occasionally the backup admin might report: this tape is unreadable, the software doesn’t recognise it. It’s very rare and I can take a look at the tape and yet I can’t see anything physically wrong with it, but for some reason they can’t use it.”
“We’ve one client who sent us a bunch of really old tapes and we have had quite a few failures with those. You don’t spend too much time on this as it’s easier to use a fresh tape. Occasionally we encounter mechanical failures: it's been dropped, or you might have a leader displaced within the tape, or perhaps nothing obviously wrong at all. We’ve been lucky whenever this has happened, as the tape has always ‘expired’ – it’s reached the end of its retention period – so there hasn’t been any value in the data. So it’s been easy to dispose of them.”
As tapes are assigned to specific tasks they become available for use once more after the retention period expires. It’s a continuous cycle of retention and release to refresh the backup content on the tape. As Hal outlines, the regular purchase of new tapes isn’t necessarily that frequent.
“When a client environment is ‘stood up’ new tapes are usually needed and the number of tapes purchased is proportional to the data that they’re likely to be storing. You’d have an initial estimate – a larger installation could be a 100 tapes – and that could last you two years."
Feeding the machine
Tape libraries are equipped with at least one tape drive – some have several, maybe eight or more – and feature multiple slots that can house handfuls to hundreds of tapes. A critical aspect is the import/export (IE) station which is where operators retrieve backup tapes or insert expired ones for re-use. Tapes aren’t kept in the IE station; it’s just a holding bay for the movement of them while avoiding any disruption of ongoing tasks within the library, as it's not uncommon for backups to be running while tapes are being put in. Hal describes the procedure.
Quantum Scalar i40 tape library: drives at the back with tape slot magazines on the sides
“A 'smart hands' operator simply presses a few buttons on the library to get the IE station drawer open, puts the tapes in closes the door followed by a few more button presses on LCD interface. Incidentally, nobody’s scanning the tapes before they go into the library. The administrators have specified their recall, so we're confident that they’re good to go back into the libraries to be re-used. There’s an online list of those tapes that we can always check if we need to.”
The machinery inside the library itself then takes over. A robotic arm equipped with a barcode scanner goes up and down and scans the tape label and that information is relayed to the backup software, which in almost all cases is CA ARCserve. The backup administrators are data centre guys, usually working remotely for our company across three continents and they have a picture of the ARCserve installation for that particular client. They know what’s going on with the backup system at any point in time.
Obviously, the backup software has a scheduler in it so the jobs start at specific times and the software can issue alerts if there are any anomalies: backup failures, that kind of thing, they all get sent through to the administrators."
“The tapes would be in one of a number of pools, and the pools would relate to the type of backup jobs they were used for. You might have tapes that are in a daily pool and the retention period on those backups might be as low as two weeks. Over that two week period, you’ve presumably done a full backup on a different pool of tapes. Now those two weeks have elapsed, that old daily becomes of no value so you can bring that tape back from Iron Mountain and into the pool where it will be logically erased and re-used."