Linux breaks 100-Petabyte ceiling

But not the first?

  • alert
  • submit to reddit

Updated We almost forgot to mention this, but Linux recently became the first desktop OS to support enormously large file sizes. How large?

144 Petabytes, or 144,000,000,000,000,000 bytes. A Petabyte is roughly a thousand Terabytes, with a Terabyte being roughly a thousand Gigabytes, of course.

This came up in conversation when we were chatting to Andre Hedrick, who looks after the Linux IDE subsystem, in our story about Mount Rainier CDs last week. Hedrick's code exploits extensions to the ATA-133 spec, which uses 48-bit rather than 28-bit addressing. The drivers are included in the 2.4.13-ac6 kernel tree, says Andre, or alternatively you can download them from his site.

The 144 Petabyte figure is obtained by raising two to the power of 48, and multiplying it by 512. A big arse number.

Some of the big enterprise vendors have claimed to support Petabyte storage for some time, and of course BeOS has supported 18-Petabyte files for many moons now, but the new drivers comfortably put Linux in front. We haven't heard of applications that demand such large files, although our most recent expense claims come pretty close.

Hoisted by our own Petabyte?

Oops. Not only can Linux not claim to be the first desktop OS to support multi-PB file sizes, it isn't even the first Unix.

"Soren Schmidt of the FreeBSD Project has added 48-bit addressing support to the FreeBSD 4-STABLE branch," writes Bruce Simpson. "You can find the news here, and the code here (Check revision 1.114 and you will find the code).

"The date on which Soren added the 48-bit support to 5.0-CURRENT is October 6th, however, the Linux patches are dated this Monday. You can therefore clearly see that FreeBSD has had working 48 bit ATA addressing support since the beginning of October, which is as of now being grandfathered in to the 4-STABLE tree after undergoing extensive testing"

Nik Clayton adds:- "Indeed, since Andre's code consists of patches to the kernel and has (as far as I can tell from reading his website) not been integrated in to the kernel's maintained by Linus, Alan, or any of the distributions, I'd say it's a little early to claim this as 'support' in any meaningful
sense of the word."

And a note about figures:-

"You guys should really know better by now ;)

"512 * 2^48 = 144,115,188,075,855,872 bytes, which is exactly 128 Petabytes," points out Jostein Trondal (along with several hundred of you too).

Finally The Beast has supported 64bit addressing in NTFS, which you can reada bout here:-

"MS does recommend a practical limit of 2 terabytes, just because of backup/restore/disaster recovery implications. Don't forget that the Terraserver mapping thing had a multi-terabyte file system, and that was yonks ago," writes a friendly regular.

Thanks to all who wrote in. ®


Designing a defence for mobile apps
In this whitepaper learn the various considerations for defending mobile applications; from the mobile application architecture itself to the myriad testing technologies needed to properly assess mobile applications risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.