Feeds

Stamen punts new approach to data aggregation

Let the data choose the questions

Choosing a cloud hosting partner with confidence

Etech: "Give me everything," says Tom Carden, a programmer and designer at Stamen Design. Stamen takes huge amounts of data and turns them into images you can interact with. Let the data choose the questions, he said, rather than the other way around.

Carden's work is part of a trend at this year's Etech emerging technology conference in San Diego, toward aggregating huge amounts of data into something meaningful.

Since Google has the most data flowing through its coffers, it's logical that Google has two aggregating experiments.

The first is prediction markets. Bo Cowgill of Google's economics group has about 1,500 Googlers signed up to bet on things like the dates of product launches. The biggest factor in getting people to bet in synch with each other is that they sit close to each other. The key finding: people tend, optimistically, to overprice things that are good for Google.

Google research director Peter Norvig has been experimenting with improving translation and spell-checking engines by throwing the web at them. Give a word processor a dictionary and all it knows is what's in that dictionary. Tell it to look around the web for samples, and colleague Merhan Sahami stops being "corrected" to "Tehran Salami".

Doing spelling is a little trickier, because so many people on the web can't spell consistently. For that, Norvig invokes probability theory. Using Bayesian rules for language applications was first proposed in 1990 at IBM, but "they didn't have the data and the computing power".

But Norvig's chief interest is practical. A theory-based approach might tell you to look for your keys under the nearest lamp post because that's where the light is brightest. Like Carden, he wants to let what works define the theory.

Carden's samples included a MySociety project that remaps the London tube according to travel time. A more complex version plots travel time to the BBC's Television Centre and house prices. Slide the bar at the top to choose your house price and travel time and see where to live. (Imagine this technique applied to tune the application of probabilities to Norvig's language efforts.)

Carden wants it all: the location of every taxi in San Francisco, all the crimes in Oakland, all the Diggs in a day. The point he says, is exploration.

"Good visualization asks questions. How many things, how many categories. As soon as you make a visualization to answer those questions, you create new ones to ask and explore. To say visualization has to answer questions would be to miss the trick of looking at the data in the first place. If you think you already know the answer you can make visualization that gives you the answer."

If, instead, you explore the pattern and see what questions it generates, you're doing something more like science.

"People tend to come to us with vast amounts of information already," he said. "Most spreadsheets are brilliant only because everyone sees the whole of the data and people can look for what they want, even if it's not in the chart."

Analytics and business intelligence, by contrast, "often start by hiding too much". ®

Internet Security Threat Report 2014

More from The Register

next story
Preview redux: Microsoft ships new Windows 10 build with 7,000 changes
Latest bleeding-edge bits borrow Action Center from Windows Phone
Google opens Inbox – email for people too thick to handle email
Print this article out and give it to someone tech-y if you get stuck
Microsoft promises Windows 10 will mean two-factor auth for all
Sneak peek at security features Redmond's baking into new OS
UNIX greybeards threaten Debian fork over systemd plan
'Veteran Unix Admins' fear desktop emphasis is betraying open source
Entity Framework goes 'code first' as Microsoft pulls visual design tool
Visual Studio database diagramming's out the window
Google+ goes TITSUP. But WHO knew? How long? Anyone ... Hello ...
Wobbly Gmail, Contacts, Calendar on the other hand ...
DEATH by PowerPoint: Microsoft warns of 0-day attack hidden in slides
Might put out patch in update, might chuck it out sooner
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.