Feeds

Yahoo!'s 'interwebs MySQL' crunches four years of Twitter

Ask the InfoChimp

Remote control for virtualized desktops

Velocity Yahoo! has plugged its YQL web query language into a third-party API that lets developers access and analyze a sea of Twitter data dating back to 2006.

"This is the real strength of YQL," Yahoo! technical evangelist Tom Hughes-Croucher told The Reg this morning at the net-infrastructure-obsessed Velocity conference in Santa Clara, California. "It's not that YQL does everything. It's that we make it easier to use services that do the things you want."

YQL — the Yahoo! Query Language — is an umbrella API that lets app developers query, filter, and join data across disparate web services offered up by Yahoo! and the web at large. InfoChimps — yes, InfoChimps, a startup based on Austin, Texas — recently introduced a beta Query API that includes several calls for crunching old Twitter data, and Yahoo! has teamed with the startup to provide a Chimpified YQL interface.

InfoChimp offers, for instance, a call dubbed Trstrank, which uses an algorithm "similar to Google PageRank" that attempts to determine the influence of a particular Twitter user. "A developer could use YQL to generate search results from Twitter, pass them through a call...to order them...and reveal not just what's being said on Twitter, but what's being said by the Twitter big guns," InfoChimps' Sarah Nordquist said of the new YQL interface in a blog post.

As the name implies, YQL mimics MySQL. "Rather than thinking about specific databases, we're thinking about an SQL-like language that treats the internet as one big database," YQL product lead Jonathan Trevor previously told The Reg. But as Hughes-Croucher tells us, there are times when the analogy breaks down. "It's really like SQL, but there are a few differences because we're dealing with web services," he said. "We can't, say, do joins unless the web service supports it."

But the language can enable joins by plugging into services like InfoChimp's. According to Hughes-Croucher, InfoChimp has moved all sorts of data — including Twitter data and US census data — onto number-crunching Hadoop clusters, and then, with YQL, you can make calls that let you tap pre-crunched data. "They'll let you do queries without hosting the data yourself," he said, "and we give you an interface in order to plug into that system."

The YQL API is available as its own web service, and the service includes a web console here (sign-in required), where you can browse example queries and test your own. YQL now offers access to about 800 data tables, including data from Yahoo! services like Flickr, plus The New York Times, Facebook, Yelp, Microsoft Bing, and Twitter. ®

Secure remote control for conventional and virtual desktops

More from The Register

next story
PEAK APPLE: iOS 8 is least popular Cupertino mobile OS in all of HUMAN HISTORY
'Nerd release' finally staggers past 50 per cent adoption
Microsoft to bake Skype into IE, without plugins
Redmond thinks the Object Real-Time Communications API for WebRTC is ready to roll
Microsoft promises Windows 10 will mean two-factor auth for all
Sneak peek at security features Redmond's baking into new OS
Mozilla: Spidermonkey ATE Apple's JavaScriptCore, THRASHED Google V8
Moz man claims the win on rivals' own benchmarks
Yes, Virginia, there IS a W3C HTML5 standard – as of now, that is
You asked for it! You begged for it! Then you gave up! And now it's HERE!
FTDI yanks chip-bricking driver from Windows Update, vows to fight on
Next driver to battle fake chips with 'non-invasive' methods
DEATH by PowerPoint: Microsoft warns of 0-day attack hidden in slides
Might put out patch in update, might chuck it out sooner
Ubuntu 14.10 tries pulling a Steve Ballmer on cloudy offerings
Oi, Windows, centOS and openSUSE – behave, we're all friends here
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.
Simplify SSL certificate management across the enterprise
Simple steps to take control of SSL across the enterprise, and recommendations for a management platform for full visibility and single-point of control for these Certificates.