Feeds

Stamen punts new approach to data aggregation

Let the data choose the questions

HP ProLiant Gen8: Integrated lifecycle automation

Etech: "Give me everything," says Tom Carden, a programmer and designer at Stamen Design. Stamen takes huge amounts of data and turns them into images you can interact with. Let the data choose the questions, he said, rather than the other way around.

Carden's work is part of a trend at this year's Etech emerging technology conference in San Diego, toward aggregating huge amounts of data into something meaningful.

Since Google has the most data flowing through its coffers, it's logical that Google has two aggregating experiments.

The first is prediction markets. Bo Cowgill of Google's economics group has about 1,500 Googlers signed up to bet on things like the dates of product launches. The biggest factor in getting people to bet in synch with each other is that they sit close to each other. The key finding: people tend, optimistically, to overprice things that are good for Google.

Google research director Peter Norvig has been experimenting with improving translation and spell-checking engines by throwing the web at them. Give a word processor a dictionary and all it knows is what's in that dictionary. Tell it to look around the web for samples, and colleague Merhan Sahami stops being "corrected" to "Tehran Salami".

Doing spelling is a little trickier, because so many people on the web can't spell consistently. For that, Norvig invokes probability theory. Using Bayesian rules for language applications was first proposed in 1990 at IBM, but "they didn't have the data and the computing power".

But Norvig's chief interest is practical. A theory-based approach might tell you to look for your keys under the nearest lamp post because that's where the light is brightest. Like Carden, he wants to let what works define the theory.

Carden's samples included a MySociety project that remaps the London tube according to travel time. A more complex version plots travel time to the BBC's Television Centre and house prices. Slide the bar at the top to choose your house price and travel time and see where to live. (Imagine this technique applied to tune the application of probabilities to Norvig's language efforts.)

Carden wants it all: the location of every taxi in San Francisco, all the crimes in Oakland, all the Diggs in a day. The point he says, is exploration.

"Good visualization asks questions. How many things, how many categories. As soon as you make a visualization to answer those questions, you create new ones to ask and explore. To say visualization has to answer questions would be to miss the trick of looking at the data in the first place. If you think you already know the answer you can make visualization that gives you the answer."

If, instead, you explore the pattern and see what questions it generates, you're doing something more like science.

"People tend to come to us with vast amounts of information already," he said. "Most spreadsheets are brilliant only because everyone sees the whole of the data and people can look for what they want, even if it's not in the chart."

Analytics and business intelligence, by contrast, "often start by hiding too much". ®

Top three mobile application threats

More from The Register

next story
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
HIDDEN packet sniffer spy tech in MILLIONS of iPhones, iPads – expert
Don't panic though – Apple's backdoor is not wide open to all, guru tells us
Mozilla fixes CRITICAL security holes in Firefox, urges v31 upgrade
Misc memory hazards 'could be exploited' - and guess what, one's a Javascript vuln
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
Captain Kirk sets phaser to SLAUGHTER after trying new Facebook app
William Shatner less-than-impressed by Zuck's celebrity-only app
Cheer up, Nokia fans. It can start making mobes again in 18 months
The real winner of the Nokia sale is *drumroll* ... Nokia
EU dons gloves, pokes Google's deals with Android mobe makers
El Reg cops a squint at investigatory letters
Chrome browser has been DRAINING PC batteries for YEARS
Google is only now fixing ancient, energy-sapping bug
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.