Spectra unveils top secret tape-based object store Black Pearl
Boulder bods about to set sail – you heard it here first
Spectra Logic is introducing a tape-based object store which uses its Deep Simple Storage Services interface and its mysterious Black Pearl appliance.
The thinking is that tape has to change from being a target for expensive and complex middleware to becoming just another storage medium. Using a flash cache with object-storing software is the way to do it.
Spectra is announcing a new interface to modernise tape library usage. Its tape libraries will be accessed via:
- Traditional backup using files
- Active archiving of files
- Storing data in buckets
Bucket storage is accomplished with Spectra's Deep Simple Storage Services, DS3, which is Amazon's S3 plus four extra commands. Think of the buckets as collections of data plus metadata. Buckets are designed to optimise the use of tape, whereas sending folders to tape, with LTFS, doesn't optimise its use as most folders are smaller than a tape's capacity.
A weather model bucket might contain, for example, all the information known about Hurricane Katrina.
There is a DS3 plug-in for Hadoop, and one of Hadoop's three file copies could be written to a Spectra archive as a bucket.
Spectra says DS3 access will be added to its disk arrays over time. Molly Rector, Spectra's chief marketing officer, said DS3 is not Fibre Channel, not a historic tape access format and not proprietary - like StorNext.
DS3 is a protocol and needs software to read and write with it. That's where Black Pearl comes in.
Jack Sparrow's BlackPearl
The Black Pearl appliance is a solid state tape library cache which turns a tape library into an object store. It is a 2U server with software running inside a virtual machine in server. This software provides tape library management, and will look after media management, such as migrating from LTO-6 to a future format such as LTO-7.
SSDs are used to cache data and Fibre Channel inks the appliance to a back-end Spectra Logic tape library.
This appliance combines files and folders into buckets. These are flatter than existing file/folder systems with their nested folders, and buckets could hold trillions of files. A bucket is an object and can store objects within itself. An object can store a file or lots of files.
Files and/or folders are sent to Black Pearl and cached in the SSDs until a bucket is full. Then the bucket is written to tape and the cycle starts again.
Rector says that disk-based object storage suppliers could use DS3 to write their objects to tape, and points out: "Object stores don't come close to our cost/GB."
The Black Pearl appliance and DS3 change tape from being a target for specialist software and specialists to being a product for everybody. Rector said: "Nobody is getting trained to write software for tape libraries using existing interfaces."
With the BlackPearl/DS3 combination tape becomes just another storage medium: "All the tape complexity goes away into the library robot."
Tapes can offload filers and become a store for old files, for the web, for archives and objects. It can be used, Rector says, in the private cloud market.
And nTier Verde?
Where does Spectra's nTier Verde disk array fit in? Why, it can run the Black Pearl software, of course, pushing data through it and DS3 to a back-end tape object store, thus provide a spinning and reeling 2-tier archival storage system.
The nTier Verde news. from last month is now revealed for what it was, a building block helping construct a combined disk and tape archive infrastructure with tiering from disk - nearline archive - to tape - offline archive.
The nTier Verde operating system is a port of ZFS to BSD.
Rector says that customers with Black Pearl and a Spectra tape library "can own their own [Amazon] Glacier."
She says Black Pearl makes tape libraries eminently suitable for big data and Spectra has some very big customers, like Yahoo!, the Discovery Channel and Nascar.
Spectra is announcing a DS3 developers' program with a downloadable simulator. We haven't seen BlackPearl pricing and availability information yet. ®