Feeds

The Long Tail's maths begin to crumble

What a coccyx!

  • alert
  • submit to reddit

3 Big data security analytics techniques

Comment "Because my background is physics and economics, I tend to go for data rather than speculation and hand waving," Chris Anderson told an interviewer recently. [ * ]

Anderson's Long Tail looks like a hit - and has already achieved the first stage of glory that buzzword-based marketing book authors aspire to - being mentioned on daytime TV. Many authors dream of this, but few achieve it, and the "long tail" is on its way to becoming this year's "surface area" or "tipping point". Or so we thought.

But now that the book is out, and Anderson's figures are under some critical scrutiny, he's engaging in quite a flurry of speculation and hand waving.

The subtitle of The Long Tail is "How Endless Choice Is Creating Unlimited Demand", but his numbers don't appear to justify such hyperbole.

There's a knack to getting the buzzword book formula right. The idea can't be false - but it shouldn't be falsifiable either. The idea should be simple enough to be explained by the subtitle, but vague enough to suggest applicability to all kinds of different markets - like a bland, all-occasions scent. And it must flatter the reader into thinking they are cleverer, and much more special than they were when they began reading the book.

And for the most part, Anderson succeeds. There is a Tail. But it's over the beguiling idea that misses can be profitably aggregated, where it begins to falter. And as for justifying the subtitle, as Anderson tells it, he's wildly overstated his case.

Slow selling products have always been successfully aggregated, and in an era when distribution or storage costs can be lowered because the stock no longer consists of physical product, or these costs can be passed on to the consumer, it's even more true. The internet, we've known for some time, allows buyers to find sellers much more easily. For music lovers, this has resulted in a resurgence of the independent labels (who pulled off a historic antitrust victory two weeks ago, by thwarting the EMI merger in a European Court), and a golden era for small label reissues. It's also a boom time for purveyors of specialist pornography - and how much more interesting The Long Tail would have been if it had tackled the economics of digital filth.

The real question raised by Anderson's 2004 essay for Wired magazine, but not answered by his book, is not whether the "tail" exists, or even how big it is, but "where can it be profitably mined?"

But now economists suggest that Anderson's more dramatic extrapolations - for example, "hits matter less" - are barely supported by the evidence he cites, and for this we have the Wall Street Journal's Lee Gomes to thank.

Anderson's initial Long Tail essay claimed that 57 per cent of Amazon.com's sales lie beyond the 100,000 books that make up a typical book store's physical inventory. This startling claim was revised downward, because the data was "funky". The published book now states "about a quarter of Amazon's book sales come from outside its top 100,000 titles."

Lee Gomes' piece entitled It May Be a Long Time Before the Long Tail Is Wagging the Web shows there are other examples that demonstrate power tipping in the wrong direction, wrong if you're a Long Tail true believer, to the best-sellers. eCast, quoted by Anderson, told Gomes that the number of songs that never get played is increasing. And, as the cost of carrying unplayed inventory rises as a proportion of total costs, the potential profitability declines. Where there's a tail, it may not be worth mining.

The Long Tail, then, isn't snake oil. Rather its restorative properties have been exaggerated. Who wants to let facts get in the way of a buzzword bestseller good thesis?

Anderson's response to Gomes' analysis has been surprisingly brittle, and quite possibly dishonest, Gomes replies.

For Harvard professor Anita Elberse, who consulted on the book, there's little more evidence than a "slight shift".

"MIT's Erik Brynjolfsson, who told me that there indeed is a shift occurring as things move online, but it's on the order of an 80/20 distribution moving to a 70/30 one," notes Gomes.

So the Long Tail doesn't, as Trackback #1 on Anderson's response echoes, show that "supply and demand have been turned upside down."

As usual, much more interesting evidence has been ignored in the bid to invert conventional economics. And the author has become a prisoner of his own metaphor. Take digital music.

As we noted last week, what may be profitable for one model doesn't necessarily apply to another model in the same market, subtleties that Anderson saw, but ignored in his eagerness to sell us his paradigm shift.

For example, eMusic CEO David Pakman again pointed out last week that Apple's iTunes Music increases the importance of the hit: 90 per cent of revenues come from 10 per cent of the catalog.

"It's made the music industry more reliant on hits. ... If iTunes was the sole model, it would wildly exacerbate the problem," he said.

By comparison, eMusic is a subscription service, which invests heavily in editorial expertise, by employing 120 writers. Profits from the Long Tail don't just fall out of trees, and attracting subscribers to the niches and obscurities requires care and attention. Rhapsody, which is correctly cited by Anderson as an example, is a subscription service like eMusic. But stressing these important points may ruin the Panglossian thesis. As much of the music business now acknowledges, a bundle, or subscription, is the best way forward.

In Anderson's defence, he may argue that he's merely satisfying a demand for feelgood marketing books, and the true purpose of "The Long Tail" is perhaps as a psychological prop.

"The teenager who spends his weekends updating a blog that nobody reads and shooting silly videos to post on YouTube.com ... He is, as Anderson’s chapter on 'The New Producers' tells us, a valiant citizen of the long tail," the New Yorker's John Cassidy observed.

And that's who needs the comfort blanket the most.

Metaphors of heads and tails can also lead us in foolish directions. Segmenting the market into two leaves no room for a "middle", and yet it's from the midlists and indies that much of our cultural richness flows. Despite the success of the indies mentioned above, it's far from clear that the net is creating as many small, sustainable businesses here as it is destroying. For the Long Tail and its true believers, who have a tendency to devalue expertise and creativity, this is heresy - amateurs will surely fill the gap.

Here's an economics question we'd love to see tackled in the successor to "The Long Tail", for there must surely be one. Do False Hopes have an opportunity cost? Do Bad Metaphors, too?®

Bootnote: For a facts and figures man who is wary of speculation, Anderson can be quite reckless. The Wired editor lost a wager with your reporter recently, which he made three years ago. Anderson bet that by spring of 2006 the sales of WLAN chipsets would exceed sales of cellular (GSM and CDMA) chipsets. He fell short by around a billion units.

Related links

Gomes
Anderson
Gomes (via Carr)

High performance access to file storage

More from The Register

next story
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
Audio fans, prepare yourself for the Second Coming ... of Blu-ray
High Fidelity Pure Audio – is this what your ears have been waiting for?
Did a date calculation bug just cost hard-up Co-op Bank £110m?
And just when Brit banking org needs £400m to stay afloat
MtGox chief Karpelès refuses to come to US for g-men's grilling
Bitcoin baron says he needs another lawyer for FinCEN chat
Zucker punched: Google gobbles Facebook-wooed Titan Aerospace
Up, up and away in my beautiful balloon flying broadband-bot
Apple DOMINATES the Valley, rakes in more profit than Google, HP, Intel, Cisco COMBINED
Cook & Co. also pay more taxes than those four worthies PLUS eBay and Oracle
It may be ILLEGAL to run Heartbleed health checks – IT lawyer
Do the right thing, earn up to 10 years in clink
prev story

Whitepapers

Top three mobile application threats
Learn about three of the top mobile application security threats facing businesses today and recommendations on how to mitigate the risk.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
SANS - Survey on application security programs
In this whitepaper learn about the state of application security programs and practices of 488 surveyed respondents, and discover how mature and effective these programs are.