Original URL: https://www.theregister.com/2008/07/31/google_cuil_spammers_analysis/
Spammers, Cuil, and the rescue from planet Google
Out-Googling Google is not an option
Posted in Software, 31st July 2008 11:02 GMT
Analysis Plenty of digital ink has been needlessly spilt this week over the launch of the suicidally-monikered new search engine Cuil.com. But the only threat to Google is itself and, in a roundabout way, the legion of spammers and "search engine optimisation" (SEO) consultants that buttress its dominance.
These dog days of summer prompted feverish hacks to ponder whether the web empire of the mighty Google might be threatened by its former employees' efforts. Ex-Googlers! Bigger index! Unusual presentation! - reports recorded in a cacophony of utter pointlessness.
In fairness, most concluded that Cuil would not be a "Google Killer". But the pondering over its chance would be laughable if it weren't so boring. We offered brief coverage of the launch here (really NSFW), to note that the results are a bit rubbish, and in some cases obscene. Our second report lifts the lid on the company's cake budget.
Romancing the Google
With a little thought, Cuil not being as good as Google at finding what we want online is the least surprising piece of news since people familiar with the situation said JPII was partial to fish on a Friday. In 2008, Mountain View's all-seeing algorithms in many ways are the web.
It's easy to identify what happened. When it first surfaced in 1998, Google made sense of the web a bit better than anyone else. It was a useful improvement on existing services. Ten years later, the web does its best to make sense of Google.
The sorry upshot is that barring some unimaginable technological leap no search engine's results will ever be better than Google's, at least in the West. And the switch leaves the likes of Microsoft and Cuil (and a dozen other doomed start-ups) effectively attempting to reverse-engineer Google, not understand the information on the web.
Microsoft's recent purchase of natural language search start-up Powerset should provide a case in point. Like many others hungry for a slice of the hugely profitable contextual search advertising business Google has created, Powerset's founders are betting web users want to ask search engines real grammatical questions.
But keyword searches, which the vast majority are adept at now, are faster for users. So the web has adapted to attract their, and Google's, attention.
Other People's Google
The people at the vanguard of reverse-engineering Google are not its jealous search rivals. They're the spammers and SEO consultants. They have driven an ever-closer relationship between the quirks and whims of Google's algorithms and policies, and the structure and content of the web. It's a feedback loop that was unavoidable once Google's early rivals proved unable to respond to its better search results and presentation.
Web spammers are looking for an audience. As the most popular search engine, Google controls what its audience sees, so the junk jockeys generate their pages in ways that game the system. Everyone else legitimately seeking those same eyeballs for their content or customers for their business want the better search rankings too, so the SEO crowd works to make legit sites dance to Google's tune. Techniques such as adding needless internal links, creating PageRank-friendly URLs and distorting normal grammar are all widely deployed with varying degrees of dastardliness.
And so it goes. Gradually the structure and content of the web becomes at one with the Google data centre. Disrupting such a tight, interconnected mutualism seems impossible for would-be "Google Killers". The best others can hope for is to imitate Google results.
Throw Google from the Train
Hitwise says Google's market share of US web searches in June was more than 69 per cent. Thanks to the mutualistic process driven by spammers and SEO consultants, that dominance is only going to increase, and it's the only "Google Killer" on the horizon.
There will be other players in the game five years from now, skiffling along on scraps where there's a lazy audience (see Microsoft's deal to supply Facebook with web search facilities, or Live.com bloatware on new PCs).
That of course is when the favours spammers and SEO consultants have been doing for Larry and Sergey will become dangerous, anti-trust style. Regulatory intervention now seems the only bar to a complete Google autocracy over the web economy.
Now is probably a good time to abandon any hope of a benevolent Google dictatorship. In toughening economic times, we're already seeing the firm being tempted to abuse its immense power to prevent its halo slipping in the eyes of Wall Street. "Do no evil" became a bad joke long ago, but its latest move to multiply its customers' search advertising costs on the sly by co-opting them into its new "Ad Matching" programme is low by any standard.
But don't blame Google's executives. It's just what monopolies do.
The spammers can help by increasing the sophistication of their junk, prompting more secret factors to be added to the byzantine Google algorithm. Kosher sites must then respond, further tailoring themselves to Mountain View's new rules, and accelerating the decline of its weak web search rivals. The at some point governments and regulators will have to act, won't they? ®