Feeds

Linux breaks 100-Petabyte ceiling

But not the first?

  • alert
  • submit to reddit

Updated We almost forgot to mention this, but Linux recently became the first desktop OS to support enormously large file sizes. How large?

144 Petabytes, or 144,000,000,000,000,000 bytes. A Petabyte is roughly a thousand Terabytes, with a Terabyte being roughly a thousand Gigabytes, of course.

This came up in conversation when we were chatting to Andre Hedrick, who looks after the Linux IDE subsystem, in our story about Mount Rainier CDs last week. Hedrick's code exploits extensions to the ATA-133 spec, which uses 48-bit rather than 28-bit addressing. The drivers are included in the 2.4.13-ac6 kernel tree, says Andre, or alternatively you can download them from his site.

The 144 Petabyte figure is obtained by raising two to the power of 48, and multiplying it by 512. A big arse number.

Some of the big enterprise vendors have claimed to support Petabyte storage for some time, and of course BeOS has supported 18-Petabyte files for many moons now, but the new drivers comfortably put Linux in front. We haven't heard of applications that demand such large files, although our most recent expense claims come pretty close.

Hoisted by our own Petabyte?

Oops. Not only can Linux not claim to be the first desktop OS to support multi-PB file sizes, it isn't even the first Unix.

"Soren Schmidt of the FreeBSD Project has added 48-bit addressing support to the FreeBSD 4-STABLE branch," writes Bruce Simpson. "You can find the news here, and the code here (Check revision 1.114 and you will find the code).

"The date on which Soren added the 48-bit support to 5.0-CURRENT is October 6th, however, the Linux patches are dated this Monday. You can therefore clearly see that FreeBSD has had working 48 bit ATA addressing support since the beginning of October, which is as of now being grandfathered in to the 4-STABLE tree after undergoing extensive testing"

Nik Clayton adds:- "Indeed, since Andre's code consists of patches to the kernel and has (as far as I can tell from reading his website) not been integrated in to the kernel's maintained by Linus, Alan, or any of the distributions, I'd say it's a little early to claim this as 'support' in any meaningful
sense of the word."

And a note about figures:-

"You guys should really know better by now ;)

"512 * 2^48 = 144,115,188,075,855,872 bytes, which is exactly 128 Petabytes," points out Jostein Trondal (along with several hundred of you too).

Finally The Beast has supported 64bit addressing in NTFS, which you can reada bout here:-

"MS does recommend a practical limit of 2 terabytes, just because of backup/restore/disaster recovery implications. Don't forget that the Terraserver mapping thing had a multi-terabyte file system, and that was yonks ago," writes a friendly regular.

Thanks to all who wrote in. ®

Whitepapers

Why cloud backup?
Combining the latest advancements in disk-based backup with secure, integrated, cloud technologies offer organizations fast and assured recovery of their critical enterprise data.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
How to determine if cloud backup is right for your servers
Two key factors, technical feasibility and TCO economics, that backup and IT operations managers should consider when assessing cloud backup.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
How to simplify SSL certificate management
Simple steps to take control of SSL certificates across the enterprise, and recommendations centralizing certificate management throughout their lifecycle.