Amazon invites 5 terabyte mondo-files into the heavens
Time to stream your genome sequencer
Magic Quadrant for Enterprise Backup/Recovery
Amazon has increased the maximum object size on its S3 online storage service to 5 terabytes. Previously, S3 users were forced to store large files in chunks no larger than about 5 gigabytes.
"When a customer wanted to access a large file or share it with others, they would either have to use several URLs in Amazon S3 or stitch the file back together using an intermediate server or within an application," Amazon said in a Friday blog post. "No more. We've raised the limit by three orders of magnitude."
Each S3 object can now range from one byte to 5 terabytes, letting you store extremely large files – including scientific or medical data, high-resolution video, and backup files – as single objects.
You can upload these larger objects using the relatively new Multipart Upload API, which was previously used to upload beefy files in parts.
OpenStack – the (truly) open source project that lets you mimic Amazon's S3 and EC2 services inside your own data center – says that it's working on larger-object support as well. OpenStack community manager Bret Piatt of Rackspace tells us that this will arrive early next year with OpenStack's "Bexar" release and that it too will expand sizes to 5 terabytes.
OpenStack was founded by Rackspace and NASA after both outfits were struggling to scale up their infrastructure clouds. OpenStack is based on Nova, a cloud fabric controller designed by NASA, and Cloud Files, a storage controller built by Rackspace. The storage platform is known as swift, and Rackspace says that it now has a "mature" swift codebase running in a production environment.
Incidentally, Bexar is named for a county in Texas, Rackspace's home state. We're told it's pronounced "bear." We would make fun of this, but we're also told that messin' with Texas is verboten. ®
COMMENTS
Pronunciation
"Incidentally, Bexar is named for a county in Texas, Rackspace's home state. We're told it's pronounced "bear." "
So a word like "Texas" would be pronounced "Teas" ?
try again
Redo your math and take into consideration simulations replication between data centers with no extra thought or effort. Then do your math again to see what it would take to replicate your budget stuff across facilities that span multiple continents.
It's easy to peg AWS as "expensive" if you are not honest about the true cost of (multiple) facilities, power, cooling, administrative burden, hardware failure/replacement, staff salary, the diesel in your generator tanks, etc. etc.
The people who claim AWS is super expensive seem to live in a fairyland where staff salaries are zero, operational burden non-existent and the cost of electricity, facilities, bandwidth and cooling is zero.
Cloud stuff is a numbers game, it's very easy to game things to fit whatever scenario you want to 'win'. I get into these depressing numbers games all the time unfortunately.
AWS is not a win for many situations but to have an honest answer you need to start with honest assumptions about the true cost of delivering these services locally.
IT infrastructure monitoring strategies
Requirements Checklist for Choosing a Cloud Backup and Recovery Service Provider
Data control in the cloud
Cloud based data management
Enabling efficient data center monitoring
