Feeds

User-generated reviews - blessing or bull?

Crowd wisdom v mob stupidity

Combat fraud and increase customer satisfaction

Analysis Last week, connectivity-hardware maker Belkin admitted that one of its employees had been using Amazon's Mechanical Turk hiring service to pay for positive - and false - reviews of Belkin products.

While we congratulate Belkin for quickly admitting its employee's unethical behavior, and while we can only assume that said employee - identified by Engadget as bizdev rep Michael Bayard - has either joined the growing ranks of the unemployed or is now on a very short leash, the entire affair raises a larger question: How reliable are user-generated reviews, anyway?

More specifically, how can an online reviews service ensure that the product evaluations its users contribute aren't tainted by self-serving bias?

After speaking with spokesfolks from a selection of online reviews sites, we've concluded that there's no way user-generated reviews can be guaranteed 100 per cent accurate.

But we've also concluded that this is OK. Sorta.

Wisdom of crowds or stupidity of mobs?

User-generated content (UGC) is the bedrock of Web 2.0. Videos on YouTube, photos on Flickr, recipes on Recipezaar ("Choose from 341,000 member-contributed recipes") - the intertubes are stuffed with the creativity, considered commentary, and crackpot craziness of literally millions of UGCers.

Such participation is all well and good. But when you want an authoritative product recommendation before making a purchase, which would you prefer as your guide - expert analysis from an experienced reviewer or a consensus opinion derived from "The Wisdom of Crowds"?

Or both?

Some product evaluation websites, such as Yelp and Epinions, rely exclusively on UGC reviews. Others, most prominently Amazon, mix UCG with "Editorial Reviews". Still others - Digital Photography Review, Macworld, Consumer Reports, and many, many more - select and vet experts before publishing their often comprehensive analyses.

One clear advantage to the site choosing to host UGC reviews is cost. The Us Ging the C offer their opinions for free, while expert reviewers are paid for their time and trouble.

But there are three other less mercenary advantages to UGC reviews: breadth of coverage, the averaging of bias, and community-based expertise.

Take Yelp, for example, which publishes UGC reviews of over 20 categories of businesses and services in over 160 US cities. A single city can have thousands of reviews - San Francisco, for example, currently has 26,409 UGC reviews of everything from fondue restaurants to dog walkers. No expert review site could come close to that reach.

Such an enormous collection of reviews also provides a large UGC site with some protection against spurious reviews by their sheer volume. On Amazon, for example, JK Rowling's Harry Potter And The Goblet Of Fire has 5,180 UGC reviews. Even if you agree with the 75 reviewers who gave that book one star, it's still instructive to know that 4,402 others gave it five stars.

Then there's the intimate, streetwise smarts that come from living in a community whose offerings are being reviewed, such as those 26K-plus San Francisco reviewers. According to Stephanie Ichinose of Yelp, "Our posters take a certain amount of pride in the reviews they post, and in their community - New York, Chicago, San Francisco, and the rest."

But there's an even more important aspect of community: the desire for a UGC community to police itself to keep bad apples out of its beloved barrel. As Ichinose says, "The community self-polices themselves. If someone spots a review they think is inappropriate, they submit a note to our customer-service team."

Patty Smith of Amazon agrees, saying "Over time, the marketplace will out false reviews as 'unhelpful'," referring to how Amazon allows UGC reviewers to review not only products, but also each other.

Combat fraud and increase customer satisfaction

More from The Register

next story
Android engineer: We DIDN'T copy Apple OR follow Samsung's orders
Veep testifies for Samsung during Apple patent trial
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Batten down the hatches, Ubuntu 14.04 LTS due in TWO DAYS
Admins dab straining server brows in advance of Trusty Tahr's long-term support landing
Microsoft lobs pre-release Windows Phone 8.1 at devs who dare
App makers can load it before anyone else, but if they do they're stuck with it
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Internet-of-stuff startup dumps NoSQL for ... SQL?
NoSQL taste great at first but lacks proper nutrients, says startup cloud whiz
Windows 8.1, which you probably haven't upgraded to yet, ALREADY OBSOLETE
Pre-Update versions of new Windows version will no longer support patches
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
Red Hat to ship RHEL 7 release candidate with a taste of container tech
Grab 'near-final' version of next Enterprise Linux next week
Ditch the sync, paddle in the Streem: Upstart offers syncless sharing
Upload, delete and carry on sharing afterwards?
prev story

Whitepapers

Designing a defence for mobile apps
In this whitepaper learn the various considerations for defending mobile applications; from the mobile application architecture itself to the myriad testing technologies needed to properly assess mobile applications risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.