Feeds

User-generated reviews - blessing or bull?

Crowd wisdom v mob stupidity

HP ProLiant Gen8: Integrated lifecycle automation

Analysis Last week, connectivity-hardware maker Belkin admitted that one of its employees had been using Amazon's Mechanical Turk hiring service to pay for positive - and false - reviews of Belkin products.

While we congratulate Belkin for quickly admitting its employee's unethical behavior, and while we can only assume that said employee - identified by Engadget as bizdev rep Michael Bayard - has either joined the growing ranks of the unemployed or is now on a very short leash, the entire affair raises a larger question: How reliable are user-generated reviews, anyway?

More specifically, how can an online reviews service ensure that the product evaluations its users contribute aren't tainted by self-serving bias?

After speaking with spokesfolks from a selection of online reviews sites, we've concluded that there's no way user-generated reviews can be guaranteed 100 per cent accurate.

But we've also concluded that this is OK. Sorta.

Wisdom of crowds or stupidity of mobs?

User-generated content (UGC) is the bedrock of Web 2.0. Videos on YouTube, photos on Flickr, recipes on Recipezaar ("Choose from 341,000 member-contributed recipes") - the intertubes are stuffed with the creativity, considered commentary, and crackpot craziness of literally millions of UGCers.

Such participation is all well and good. But when you want an authoritative product recommendation before making a purchase, which would you prefer as your guide - expert analysis from an experienced reviewer or a consensus opinion derived from "The Wisdom of Crowds"?

Or both?

Some product evaluation websites, such as Yelp and Epinions, rely exclusively on UGC reviews. Others, most prominently Amazon, mix UCG with "Editorial Reviews". Still others - Digital Photography Review, Macworld, Consumer Reports, and many, many more - select and vet experts before publishing their often comprehensive analyses.

One clear advantage to the site choosing to host UGC reviews is cost. The Us Ging the C offer their opinions for free, while expert reviewers are paid for their time and trouble.

But there are three other less mercenary advantages to UGC reviews: breadth of coverage, the averaging of bias, and community-based expertise.

Take Yelp, for example, which publishes UGC reviews of over 20 categories of businesses and services in over 160 US cities. A single city can have thousands of reviews - San Francisco, for example, currently has 26,409 UGC reviews of everything from fondue restaurants to dog walkers. No expert review site could come close to that reach.

Such an enormous collection of reviews also provides a large UGC site with some protection against spurious reviews by their sheer volume. On Amazon, for example, JK Rowling's Harry Potter And The Goblet Of Fire has 5,180 UGC reviews. Even if you agree with the 75 reviewers who gave that book one star, it's still instructive to know that 4,402 others gave it five stars.

Then there's the intimate, streetwise smarts that come from living in a community whose offerings are being reviewed, such as those 26K-plus San Francisco reviewers. According to Stephanie Ichinose of Yelp, "Our posters take a certain amount of pride in the reviews they post, and in their community - New York, Chicago, San Francisco, and the rest."

But there's an even more important aspect of community: the desire for a UGC community to police itself to keep bad apples out of its beloved barrel. As Ichinose says, "The community self-polices themselves. If someone spots a review they think is inappropriate, they submit a note to our customer-service team."

Patty Smith of Amazon agrees, saying "Over time, the marketplace will out false reviews as 'unhelpful'," referring to how Amazon allows UGC reviewers to review not only products, but also each other.

Top three mobile application threats

More from The Register

next story
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
HIDDEN packet sniffer spy tech in MILLIONS of iPhones, iPads – expert
Don't panic though – Apple's backdoor is not wide open to all, guru tells us
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
Captain Kirk sets phaser to SLAUGHTER after trying new Facebook app
William Shatner less-than-impressed by Zuck's celebrity-only app
Do YOU work at Microsoft? Um. Are you SURE about that?
Nokia and marketing types first to get the bullet, says report
Microsoft takes on Chromebook with low-cost Windows laptops
Redmond's chief salesman: We're taking 'hard' decisions
Cheer up, Nokia fans. It can start making mobes again in 18 months
The real winner of the Nokia sale is *drumroll* ... Nokia
EU dons gloves, pokes Google's deals with Android mobe makers
El Reg cops a squint at investigatory letters
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.