Feeds

Linux breaks 100-Petabyte ceiling

But not the first?

  • alert
  • submit to reddit

Updated We almost forgot to mention this, but Linux recently became the first desktop OS to support enormously large file sizes. How large?

144 Petabytes, or 144,000,000,000,000,000 bytes. A Petabyte is roughly a thousand Terabytes, with a Terabyte being roughly a thousand Gigabytes, of course.

This came up in conversation when we were chatting to Andre Hedrick, who looks after the Linux IDE subsystem, in our story about Mount Rainier CDs last week. Hedrick's code exploits extensions to the ATA-133 spec, which uses 48-bit rather than 28-bit addressing. The drivers are included in the 2.4.13-ac6 kernel tree, says Andre, or alternatively you can download them from his site.

The 144 Petabyte figure is obtained by raising two to the power of 48, and multiplying it by 512. A big arse number.

Some of the big enterprise vendors have claimed to support Petabyte storage for some time, and of course BeOS has supported 18-Petabyte files for many moons now, but the new drivers comfortably put Linux in front. We haven't heard of applications that demand such large files, although our most recent expense claims come pretty close.

Hoisted by our own Petabyte?

Oops. Not only can Linux not claim to be the first desktop OS to support multi-PB file sizes, it isn't even the first Unix.

"Soren Schmidt of the FreeBSD Project has added 48-bit addressing support to the FreeBSD 4-STABLE branch," writes Bruce Simpson. "You can find the news here, and the code here (Check revision 1.114 and you will find the code).

"The date on which Soren added the 48-bit support to 5.0-CURRENT is October 6th, however, the Linux patches are dated this Monday. You can therefore clearly see that FreeBSD has had working 48 bit ATA addressing support since the beginning of October, which is as of now being grandfathered in to the 4-STABLE tree after undergoing extensive testing"

Nik Clayton adds:- "Indeed, since Andre's code consists of patches to the kernel and has (as far as I can tell from reading his website) not been integrated in to the kernel's maintained by Linus, Alan, or any of the distributions, I'd say it's a little early to claim this as 'support' in any meaningful
sense of the word."

And a note about figures:-

"You guys should really know better by now ;)

"512 * 2^48 = 144,115,188,075,855,872 bytes, which is exactly 128 Petabytes," points out Jostein Trondal (along with several hundred of you too).

Finally The Beast has supported 64bit addressing in NTFS, which you can reada bout here:-

"MS does recommend a practical limit of 2 terabytes, just because of backup/restore/disaster recovery implications. Don't forget that the Terraserver mapping thing had a multi-terabyte file system, and that was yonks ago," writes a friendly regular.

Thanks to all who wrote in. ®

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 10 endpoint backup mistakes
Avoid the ten endpoint backup mistakes to ensure that your critical corporate data is protected and end user productivity is improved.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Rethinking backup and recovery in the modern data center
Combining intelligence, operational analytics, and automation to enable efficient, data-driven IT organizations using the HP ABR approach.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.