Feeds

Linux breaks 100-Petabyte ceiling

But not the first?

  • alert
  • submit to reddit

Updated We almost forgot to mention this, but Linux recently became the first desktop OS to support enormously large file sizes. How large?

144 Petabytes, or 144,000,000,000,000,000 bytes. A Petabyte is roughly a thousand Terabytes, with a Terabyte being roughly a thousand Gigabytes, of course.

This came up in conversation when we were chatting to Andre Hedrick, who looks after the Linux IDE subsystem, in our story about Mount Rainier CDs last week. Hedrick's code exploits extensions to the ATA-133 spec, which uses 48-bit rather than 28-bit addressing. The drivers are included in the 2.4.13-ac6 kernel tree, says Andre, or alternatively you can download them from his site.

The 144 Petabyte figure is obtained by raising two to the power of 48, and multiplying it by 512. A big arse number.

Some of the big enterprise vendors have claimed to support Petabyte storage for some time, and of course BeOS has supported 18-Petabyte files for many moons now, but the new drivers comfortably put Linux in front. We haven't heard of applications that demand such large files, although our most recent expense claims come pretty close.

Hoisted by our own Petabyte?

Oops. Not only can Linux not claim to be the first desktop OS to support multi-PB file sizes, it isn't even the first Unix.

"Soren Schmidt of the FreeBSD Project has added 48-bit addressing support to the FreeBSD 4-STABLE branch," writes Bruce Simpson. "You can find the news here, and the code here (Check revision 1.114 and you will find the code).

"The date on which Soren added the 48-bit support to 5.0-CURRENT is October 6th, however, the Linux patches are dated this Monday. You can therefore clearly see that FreeBSD has had working 48 bit ATA addressing support since the beginning of October, which is as of now being grandfathered in to the 4-STABLE tree after undergoing extensive testing"

Nik Clayton adds:- "Indeed, since Andre's code consists of patches to the kernel and has (as far as I can tell from reading his website) not been integrated in to the kernel's maintained by Linus, Alan, or any of the distributions, I'd say it's a little early to claim this as 'support' in any meaningful
sense of the word."

And a note about figures:-

"You guys should really know better by now ;)

"512 * 2^48 = 144,115,188,075,855,872 bytes, which is exactly 128 Petabytes," points out Jostein Trondal (along with several hundred of you too).

Finally The Beast has supported 64bit addressing in NTFS, which you can reada bout here:-

"MS does recommend a practical limit of 2 terabytes, just because of backup/restore/disaster recovery implications. Don't forget that the Terraserver mapping thing had a multi-terabyte file system, and that was yonks ago," writes a friendly regular.

Thanks to all who wrote in. ®

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.