Feeds

User-generated reviews - blessing or bull?

Crowd wisdom v mob stupidity

Providing a secure and efficient Helpdesk

Analysis Last week, connectivity-hardware maker Belkin admitted that one of its employees had been using Amazon's Mechanical Turk hiring service to pay for positive - and false - reviews of Belkin products.

While we congratulate Belkin for quickly admitting its employee's unethical behavior, and while we can only assume that said employee - identified by Engadget as bizdev rep Michael Bayard - has either joined the growing ranks of the unemployed or is now on a very short leash, the entire affair raises a larger question: How reliable are user-generated reviews, anyway?

More specifically, how can an online reviews service ensure that the product evaluations its users contribute aren't tainted by self-serving bias?

After speaking with spokesfolks from a selection of online reviews sites, we've concluded that there's no way user-generated reviews can be guaranteed 100 per cent accurate.

But we've also concluded that this is OK. Sorta.

Wisdom of crowds or stupidity of mobs?

User-generated content (UGC) is the bedrock of Web 2.0. Videos on YouTube, photos on Flickr, recipes on Recipezaar ("Choose from 341,000 member-contributed recipes") - the intertubes are stuffed with the creativity, considered commentary, and crackpot craziness of literally millions of UGCers.

Such participation is all well and good. But when you want an authoritative product recommendation before making a purchase, which would you prefer as your guide - expert analysis from an experienced reviewer or a consensus opinion derived from "The Wisdom of Crowds"?

Or both?

Some product evaluation websites, such as Yelp and Epinions, rely exclusively on UGC reviews. Others, most prominently Amazon, mix UCG with "Editorial Reviews". Still others - Digital Photography Review, Macworld, Consumer Reports, and many, many more - select and vet experts before publishing their often comprehensive analyses.

One clear advantage to the site choosing to host UGC reviews is cost. The Us Ging the C offer their opinions for free, while expert reviewers are paid for their time and trouble.

But there are three other less mercenary advantages to UGC reviews: breadth of coverage, the averaging of bias, and community-based expertise.

Take Yelp, for example, which publishes UGC reviews of over 20 categories of businesses and services in over 160 US cities. A single city can have thousands of reviews - San Francisco, for example, currently has 26,409 UGC reviews of everything from fondue restaurants to dog walkers. No expert review site could come close to that reach.

Such an enormous collection of reviews also provides a large UGC site with some protection against spurious reviews by their sheer volume. On Amazon, for example, JK Rowling's Harry Potter And The Goblet Of Fire has 5,180 UGC reviews. Even if you agree with the 75 reviewers who gave that book one star, it's still instructive to know that 4,402 others gave it five stars.

Then there's the intimate, streetwise smarts that come from living in a community whose offerings are being reviewed, such as those 26K-plus San Francisco reviewers. According to Stephanie Ichinose of Yelp, "Our posters take a certain amount of pride in the reviews they post, and in their community - New York, Chicago, San Francisco, and the rest."

But there's an even more important aspect of community: the desire for a UGC community to police itself to keep bad apples out of its beloved barrel. As Ichinose says, "The community self-polices themselves. If someone spots a review they think is inappropriate, they submit a note to our customer-service team."

Patty Smith of Amazon agrees, saying "Over time, the marketplace will out false reviews as 'unhelpful'," referring to how Amazon allows UGC reviewers to review not only products, but also each other.

Reducing the cost and complexity of web vulnerability management

More from The Register

next story
'Windows 9' LEAK: Microsoft's playing catchup with Linux
Multiple desktops and live tiles in restored Start button star in new vids
Not appy with your Chromebook? Well now it can run Android apps
Google offers beta of tricky OS-inside-OS tech
New 'Cosmos' browser surfs the net by TXT alone
No data plan? No WiFi? No worries ... except sluggish download speed
iOS 8 release: WebGL now runs everywhere. Hurrah for 3D graphics!
HTML 5's pretty neat ... when your browser supports it
Greater dev access to iOS 8 will put us AT RISK from HACKERS
Knocking holes in Apple's walled garden could backfire, says securo-chap
NHS grows a NoSQL backbone and rips out its Oracle Spine
Open source? In the government? Ha ha! What, wait ...?
Google extends app refund window to two hours
You now have 120 minutes to finish that game instead of 15
Intel: Hey, enterprises, drop everything and DO HADOOP
Big Data analytics projected to run on more servers than any other app
prev story

Whitepapers

Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.
Security and trust: The backbone of doing business over the internet
Explores the current state of website security and the contributions Symantec is making to help organizations protect critical data and build trust with customers.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.