Feeds

Stamen punts new approach to data aggregation

Let the data choose the questions

5 things you didn’t know about cloud backup

Etech: "Give me everything," says Tom Carden, a programmer and designer at Stamen Design. Stamen takes huge amounts of data and turns them into images you can interact with. Let the data choose the questions, he said, rather than the other way around.

Carden's work is part of a trend at this year's Etech emerging technology conference in San Diego, toward aggregating huge amounts of data into something meaningful.

Since Google has the most data flowing through its coffers, it's logical that Google has two aggregating experiments.

The first is prediction markets. Bo Cowgill of Google's economics group has about 1,500 Googlers signed up to bet on things like the dates of product launches. The biggest factor in getting people to bet in synch with each other is that they sit close to each other. The key finding: people tend, optimistically, to overprice things that are good for Google.

Google research director Peter Norvig has been experimenting with improving translation and spell-checking engines by throwing the web at them. Give a word processor a dictionary and all it knows is what's in that dictionary. Tell it to look around the web for samples, and colleague Merhan Sahami stops being "corrected" to "Tehran Salami".

Doing spelling is a little trickier, because so many people on the web can't spell consistently. For that, Norvig invokes probability theory. Using Bayesian rules for language applications was first proposed in 1990 at IBM, but "they didn't have the data and the computing power".

But Norvig's chief interest is practical. A theory-based approach might tell you to look for your keys under the nearest lamp post because that's where the light is brightest. Like Carden, he wants to let what works define the theory.

Carden's samples included a MySociety project that remaps the London tube according to travel time. A more complex version plots travel time to the BBC's Television Centre and house prices. Slide the bar at the top to choose your house price and travel time and see where to live. (Imagine this technique applied to tune the application of probabilities to Norvig's language efforts.)

Carden wants it all: the location of every taxi in San Francisco, all the crimes in Oakland, all the Diggs in a day. The point he says, is exploration.

"Good visualization asks questions. How many things, how many categories. As soon as you make a visualization to answer those questions, you create new ones to ask and explore. To say visualization has to answer questions would be to miss the trick of looking at the data in the first place. If you think you already know the answer you can make visualization that gives you the answer."

If, instead, you explore the pattern and see what questions it generates, you're doing something more like science.

"People tend to come to us with vast amounts of information already," he said. "Most spreadsheets are brilliant only because everyone sees the whole of the data and people can look for what they want, even if it's not in the chart."

Analytics and business intelligence, by contrast, "often start by hiding too much". ®

5 things you didn’t know about cloud backup

More from The Register

next story
The Return of BSOD: Does ANYONE trust Microsoft patches?
Sysadmins, you're either fighting fires or seen as incompetents now
Munich considers dumping Linux for ... GULP ... Windows!
Give a penguinista a hug, the Outlook's not good for open source's poster child
Intel's Raspberry Pi rival Galileo can now run Windows
Behold the Internet of Things. Wintel Things
Linux Foundation says many Linux admins and engineers are certifiable
Floats exam program to help IT employers lock up talent
Microsoft cries UNINSTALL in the wake of Blue Screens of Death™
Cache crash causes contained choloric calamity
Eat up Martha! Microsoft slings handwriting recog into OneNote on Android
Freehand input on non-Windows kit for the first time
Linux kernel devs made to finger their dongles before contributing code
Two-factor auth enabled for Kernel.org repositories
prev story

Whitepapers

Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
7 Elements of Radically Simple OS Migration
Avoid the typical headaches of OS migration during your next project by learning about 7 elements of radically simple OS migration.
BYOD's dark side: Data protection
An endpoint data protection solution that adds value to the user and the organization so it can protect itself from data loss as well as leverage corporate data.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?