Feeds

Phew! These DynamoDB queries are heavy. C'mon Amazon, give us a hand

Ah, there you are, and with a fistful of more indexes

The Essential Guide to IT Transformation

Amazon has brought global secondary indexes to its non-relational, rentable DynamoDB database.

The upgrade means cloud wranglers can perform more complicated queries on data stored in the system without having to sacrifice latency. It achieves this by implementing Global Secondary Indexes (GSIs) – a feature that lets admins create additional indexes for the system.

Amazon introduced Local Secondary Indexes (LSIs) in September, which gave admins the ability to locally partition their DynamoDB instance to have up to five additional indexes to use for queries, letting them avoid costly scatter-gather operations and instead run data mining against cached indexes.

Standard DynamoDB splits a data table into attributes, which are roughly analogous to columns. The first two attributes form the hash and the range, and DynamoDB lets you query against either the hash, or the hash and the range.

This lets you fire off queries like 'Show us everything by Team Register [hash]', or using a hash key with a range key, eg 'Show me the ten most recent articles [range] by Team Register [hash]', but more complex queries are difficult.

Now both LSIs and GSIs let admins add in up to five additional indexes, so you can ask more complicated questions like 'Show us the ten most recent articles by Team Register with more than 10 comments that reference LOHAN'.

"With global secondary indexes in DynamoDB, you can now have many indexes and tune their capacity independently. These indexes also provide query/cost flexibility, allowing a custom level of clustering to be defined per index. Developers can specify which attributes should be 'projected' to the secondary index, allowing faster access to often-accessed data, while avoiding extra read/write costs for other attributes," writes Amazon's chief technology officer Werner Vogels in a blog post describing the change.

The rollout of Global Secondary Indexes follows a major price cut of the service by Amazon in May in response to the launch of Google's own Cloud Datastore, and the launch of an offline capability in September to help road-warrior developers test out apps built on the system without having to be online.

Though Amazon's storage (S3) and compute services (EC2) are reckoned to make a shedload of cash, the company has a need to get more sticky data into its cloud to protect against competitors like Windows Azure and Google Compute Engine. Though GSIs are a useful feature, they also encourage cloudy database developers to architect applications for AWS's specific way of doing things – which though not lock-in, does make the Amazon cloud harder to leave. ®

The Essential Guide to IT Transformation

More from The Register

next story
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
Microsoft says 'weird things' can happen during Windows Server 2003 migrations
Fix coming for bug that makes Kerberos croak when you run two domain controllers
Cisco says network virtualisation won't pay off everywhere
Another sign of strain in the Borg/VMware relationship?
VVOL update: Are any vendors NOT leaping into bed with VMware?
It's not yet been released but everyone thinks it's the dog's danglies
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.
Maximize storage efficiency across the enterprise
The HP StoreOnce backup solution offers highly flexible, centrally managed, and highly efficient data protection for any enterprise.