Feeds

Yahoo!'s 'interwebs MySQL' crunches four years of Twitter

Ask the InfoChimp

Boost IT visibility and business value

Velocity Yahoo! has plugged its YQL web query language into a third-party API that lets developers access and analyze a sea of Twitter data dating back to 2006.

"This is the real strength of YQL," Yahoo! technical evangelist Tom Hughes-Croucher told The Reg this morning at the net-infrastructure-obsessed Velocity conference in Santa Clara, California. "It's not that YQL does everything. It's that we make it easier to use services that do the things you want."

YQL — the Yahoo! Query Language — is an umbrella API that lets app developers query, filter, and join data across disparate web services offered up by Yahoo! and the web at large. InfoChimps — yes, InfoChimps, a startup based on Austin, Texas — recently introduced a beta Query API that includes several calls for crunching old Twitter data, and Yahoo! has teamed with the startup to provide a Chimpified YQL interface.

InfoChimp offers, for instance, a call dubbed Trstrank, which uses an algorithm "similar to Google PageRank" that attempts to determine the influence of a particular Twitter user. "A developer could use YQL to generate search results from Twitter, pass them through a call...to order them...and reveal not just what's being said on Twitter, but what's being said by the Twitter big guns," InfoChimps' Sarah Nordquist said of the new YQL interface in a blog post.

As the name implies, YQL mimics MySQL. "Rather than thinking about specific databases, we're thinking about an SQL-like language that treats the internet as one big database," YQL product lead Jonathan Trevor previously told The Reg. But as Hughes-Croucher tells us, there are times when the analogy breaks down. "It's really like SQL, but there are a few differences because we're dealing with web services," he said. "We can't, say, do joins unless the web service supports it."

But the language can enable joins by plugging into services like InfoChimp's. According to Hughes-Croucher, InfoChimp has moved all sorts of data — including Twitter data and US census data — onto number-crunching Hadoop clusters, and then, with YQL, you can make calls that let you tap pre-crunched data. "They'll let you do queries without hosting the data yourself," he said, "and we give you an interface in order to plug into that system."

The YQL API is available as its own web service, and the service includes a web console here (sign-in required), where you can browse example queries and test your own. YQL now offers access to about 800 data tables, including data from Yahoo! services like Flickr, plus The New York Times, Facebook, Yelp, Microsoft Bing, and Twitter. ®

The Essential Guide to IT Transformation

More from The Register

next story
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
KDE releases ice-cream coloured Plasma 5 just in time for summer
Melty but refreshing - popular rival to Mint's Cinnamon's still a work in progress
Leaked Windows Phone 8.1 Update specs tease details of Nokia's next mobes
New screen sizes, dual SIMs, voice over LTE, and more
Another day, another Firefox: Version 31 is upon us ALREADY
Web devs, Mozilla really wants you to like this one
Put down that Oracle database patch: It could cost $23,000 per CPU
On-by-default INMEMORY tech a boon for developers ... as long as they can afford it
Mozilla keeps its Beard, hopes anti-gay marriage troubles are now over
Plenty on new CEO's todo list – starting with Firefox's slipping grasp
Apple: We'll unleash OS X Yosemite beta on the MASSES on 24 July
Starting today, regular fanbois will be guinea pigs, it tells Reg
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Build a business case: developing custom apps
Learn how to maximize the value of custom applications by accelerating and simplifying their development.