Feeds

Stamen punts new approach to data aggregation

Let the data choose the questions

3 Big data security analytics techniques

Etech: "Give me everything," says Tom Carden, a programmer and designer at Stamen Design. Stamen takes huge amounts of data and turns them into images you can interact with. Let the data choose the questions, he said, rather than the other way around.

Carden's work is part of a trend at this year's Etech emerging technology conference in San Diego, toward aggregating huge amounts of data into something meaningful.

Since Google has the most data flowing through its coffers, it's logical that Google has two aggregating experiments.

The first is prediction markets. Bo Cowgill of Google's economics group has about 1,500 Googlers signed up to bet on things like the dates of product launches. The biggest factor in getting people to bet in synch with each other is that they sit close to each other. The key finding: people tend, optimistically, to overprice things that are good for Google.

Google research director Peter Norvig has been experimenting with improving translation and spell-checking engines by throwing the web at them. Give a word processor a dictionary and all it knows is what's in that dictionary. Tell it to look around the web for samples, and colleague Merhan Sahami stops being "corrected" to "Tehran Salami".

Doing spelling is a little trickier, because so many people on the web can't spell consistently. For that, Norvig invokes probability theory. Using Bayesian rules for language applications was first proposed in 1990 at IBM, but "they didn't have the data and the computing power".

But Norvig's chief interest is practical. A theory-based approach might tell you to look for your keys under the nearest lamp post because that's where the light is brightest. Like Carden, he wants to let what works define the theory.

Carden's samples included a MySociety project that remaps the London tube according to travel time. A more complex version plots travel time to the BBC's Television Centre and house prices. Slide the bar at the top to choose your house price and travel time and see where to live. (Imagine this technique applied to tune the application of probabilities to Norvig's language efforts.)

Carden wants it all: the location of every taxi in San Francisco, all the crimes in Oakland, all the Diggs in a day. The point he says, is exploration.

"Good visualization asks questions. How many things, how many categories. As soon as you make a visualization to answer those questions, you create new ones to ask and explore. To say visualization has to answer questions would be to miss the trick of looking at the data in the first place. If you think you already know the answer you can make visualization that gives you the answer."

If, instead, you explore the pattern and see what questions it generates, you're doing something more like science.

"People tend to come to us with vast amounts of information already," he said. "Most spreadsheets are brilliant only because everyone sees the whole of the data and people can look for what they want, even if it's not in the chart."

Analytics and business intelligence, by contrast, "often start by hiding too much". ®

Top three mobile application threats

More from The Register

next story
Next Windows obsolescence panic is 450 days from … NOW!
The clock is ticking louder for Windows Server 2003 R2 users
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Ditch the sync, paddle in the Streem: Upstart offers syncless sharing
Upload, delete and carry on sharing afterwards?
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
Batten down the hatches, Ubuntu 14.04 LTS due in TWO DAYS
Admins dab straining server brows in advance of Trusty Tahr's long-term support landing
Red Hat to ship RHEL 7 release candidate with a taste of container tech
Grab 'near-final' version of next Enterprise Linux next week
Windows 8.1, which you probably haven't upgraded to yet, ALREADY OBSOLETE
Pre-Update versions of new Windows version will no longer support patches
Inside the Hekaton: SQL Server 2014's database engine deconstructed
Nadella's database sqares the circle of cheap memory vs speed
prev story

Whitepapers

Mainstay ROI - Does application security pay?
In this whitepaper learn how you and your enterprise might benefit from better software security.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.