Microsoft reinvents Massive Arrays of Idle Disks for Azure, 'cos IBM tape ain't enough
Redmond claims to be world’s biggest tape user, but wants a faster alternative for AI data
Microsoft is working on a new class of disk-based storage appliance for its Azure cloud service, because tape is proving too hard and too slow to use in some situations.
Azure chief technology officer Mark Russinovich today told an event in Sydney, Australia, that Microsoft has deployed tape libraries to power its Azure Archive Service, but said “we are not satisfied there.” Russinovich showed a slide depicting an IBM TS3500 tape library and said “We think there is another point in the [storage cost vs. access latency] spectrum that is slightly higher cost, but much more efficient than hard disks.”
“The reason we are investing in this is that tape enclosures are fairly expensive,” he added. “They require special infrastructure in the data centre.”
Microsoft’s response is dubbed “Project Pelican” and aims to deliver a rack packed with 1,152 10TB, 3.5-inch disks, for raw capacity of 11.5 petabytes.
Russinovich said Pelican’s design also includes a couple of servers. “What makes this unique is that the disks cannot all spin up at the same time,” he explained. “When data is being accessed the server spins up the drives. This allows you to store massive amounts of data with very little power.” And because the data is on disk, access times are decent – and positively hasty compared to tape.
As described by Russinovich, Pelican sounds a lot like Massive Arrays of Idle Disks – aka MAID – a technology of which The Register’s storage desk was fond. Our ardour may have been misplaced in 2010, when the technology’s main promoter, a company called COPAN, called it quits and sold itself to SGI.
Another wonky Pelican pic: the new kit vs. the old
Perhaps Pelican is MAID redux. Or maybe not: Russinovich didn’t specify when or if Pelican might be pressed into service. But he did position it as the heir to Redmond's current cloud storage hardware. He also said that Pelican, and Azure Archive Storage, are well-suited to providing storage for the massive piles of data required for AI models. Today, Russinovich said, organisations don’t retain data that extensive analysis shows might make for better use of AI. Microsoft’s hope is that it can encourage users to hang onto all their data and pay to store it forever.
Also suggesting Pelican will fly was Russinovich’s admission that some Azure data centres aren’t ready to house tape libraries, but won’t be troubled by disk-based Pelican.
The CTO didn't signal that tape has no future at Azure, so IBM's storage team can sleep soundly. But he did point out that tape has limitations and that Microsoft wants another tier of storage. On the downside, Microsoft's discussed Pelican for a few years without putting it into production. Perhaps AI has given it impetus.
Russinovich also said Microsoft has experimented with on-site natural gas generators for some Azure data centres. The generators feed a bank of batteries that power the facilities, an arrangement he said is more efficient than using electricity from the local grid. ®