Feeds

User-generated reviews - blessing or bull?

Crowd wisdom v mob stupidity

Top 5 reasons to deploy VMware with Tegile

Analysis Last week, connectivity-hardware maker Belkin admitted that one of its employees had been using Amazon's Mechanical Turk hiring service to pay for positive - and false - reviews of Belkin products.

While we congratulate Belkin for quickly admitting its employee's unethical behavior, and while we can only assume that said employee - identified by Engadget as bizdev rep Michael Bayard - has either joined the growing ranks of the unemployed or is now on a very short leash, the entire affair raises a larger question: How reliable are user-generated reviews, anyway?

More specifically, how can an online reviews service ensure that the product evaluations its users contribute aren't tainted by self-serving bias?

After speaking with spokesfolks from a selection of online reviews sites, we've concluded that there's no way user-generated reviews can be guaranteed 100 per cent accurate.

But we've also concluded that this is OK. Sorta.

Wisdom of crowds or stupidity of mobs?

User-generated content (UGC) is the bedrock of Web 2.0. Videos on YouTube, photos on Flickr, recipes on Recipezaar ("Choose from 341,000 member-contributed recipes") - the intertubes are stuffed with the creativity, considered commentary, and crackpot craziness of literally millions of UGCers.

Such participation is all well and good. But when you want an authoritative product recommendation before making a purchase, which would you prefer as your guide - expert analysis from an experienced reviewer or a consensus opinion derived from "The Wisdom of Crowds"?

Or both?

Some product evaluation websites, such as Yelp and Epinions, rely exclusively on UGC reviews. Others, most prominently Amazon, mix UCG with "Editorial Reviews". Still others - Digital Photography Review, Macworld, Consumer Reports, and many, many more - select and vet experts before publishing their often comprehensive analyses.

One clear advantage to the site choosing to host UGC reviews is cost. The Us Ging the C offer their opinions for free, while expert reviewers are paid for their time and trouble.

But there are three other less mercenary advantages to UGC reviews: breadth of coverage, the averaging of bias, and community-based expertise.

Take Yelp, for example, which publishes UGC reviews of over 20 categories of businesses and services in over 160 US cities. A single city can have thousands of reviews - San Francisco, for example, currently has 26,409 UGC reviews of everything from fondue restaurants to dog walkers. No expert review site could come close to that reach.

Such an enormous collection of reviews also provides a large UGC site with some protection against spurious reviews by their sheer volume. On Amazon, for example, JK Rowling's Harry Potter And The Goblet Of Fire has 5,180 UGC reviews. Even if you agree with the 75 reviewers who gave that book one star, it's still instructive to know that 4,402 others gave it five stars.

Then there's the intimate, streetwise smarts that come from living in a community whose offerings are being reviewed, such as those 26K-plus San Francisco reviewers. According to Stephanie Ichinose of Yelp, "Our posters take a certain amount of pride in the reviews they post, and in their community - New York, Chicago, San Francisco, and the rest."

But there's an even more important aspect of community: the desire for a UGC community to police itself to keep bad apples out of its beloved barrel. As Ichinose says, "The community self-polices themselves. If someone spots a review they think is inappropriate, they submit a note to our customer-service team."

Patty Smith of Amazon agrees, saying "Over time, the marketplace will out false reviews as 'unhelpful'," referring to how Amazon allows UGC reviewers to review not only products, but also each other.

Intelligent flash storage arrays

More from The Register

next story
Euro Parliament VOTES to BREAK UP GOOGLE. Er, OK then
It CANNA do it, captain.They DON'T have the POWER!
Download alert: Nearly ALL top 100 Android, iOS paid apps hacked
Attack of the Clones? Yeah, but much, much scarier – report
NSA SOURCE CODE LEAK: Information slurp tools to appear online
Now you can run your own intelligence agency
Post-Microsoft, post-PC programming: The portable REVOLUTION
Code jockeys: count up and grab your fabulous tablets
Twitter App Graph exposes smartphone spyware feature
You don't want everyone to compile app lists from your fondleware? BAD LUCK
Microsoft adds video offering to Office 365. Oh NOES, you'll need Adobe Flash
Lovely presentations... but not on your Flash-hating mobe
prev story

Whitepapers

10 ways wire data helps conquer IT complexity
IT teams can automatically detect problems across the IT environment, spot data theft, select unique pieces of transaction payloads to send to a data source, and more.
A strategic approach to identity relationship management
ForgeRock commissioned Forrester to evaluate companies’ IAM practices and requirements when it comes to customer-facing scenarios versus employee-facing ones.
How to determine if cloud backup is right for your servers
Two key factors, technical feasibility and TCO economics, that backup and IT operations managers should consider when assessing cloud backup.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Choosing a cloud hosting partner with confidence
Download Choosing a Cloud Hosting Provider with Confidence to learn more about cloud computing - the new opportunities and new security challenges.