Wikia snatches open search software from LookSmart
Will deliver Grub to mankind. Or whatever
OSCON Wikia, the latest effort from Wikipedia co-founder Jimbo Wales, has bought the distributed search software Grub from LookSmart for an undisclosed sum and open sourced it.
Wales, speaking here at OSCON, announced the move during a morning speech. He pitched Grub as being analogous to Seti@home in that users run the client software in the background on their PCs. Grub then indexes URLs and sends the information back to centralized servers. The software had languished under the stewardship of Looksmart, and Wales now hopes to make it a core piece of Wikia.
For the unfamiliar, Wikia is an attempt to create an open source rival to Yahoo!, Google or Microsoft.
"In search, we have a handful of proprietary places," Wales said, during his keynote. "They control the vast majority of traffic on the internet.
"Those search results are an editorial statement even if they are driven by an algorithm. We really have no idea how they are doing it. That process is kept deliberately secret."
So, Wikia will deliver open source algorithms to developers and work to create an open index of the internet. If the project ends up resembling Wikipedia, we don't hold out much hope of helpful search results.
"Together, we can all kick Google's ass," Wales said.
From where we sit, an open source search engine that turned up good results and was actually used by lots of people could end up as a very good thing. Google, in particular, has seized too much power with its vast knowledge of the world's desires. In addition, the company has not proven very able at protecting privacy on a satisfactory level.
Would Wikia be a better organization to hold such power just because its code was open? Not really, unless guarantees are made around the immediate destruction of search logs and the like.
You can find very little on Grub here. ®
First, one of the main reasons search algos are kept secret is that revealing them runs the risk of turning the index into a spammer's paradise. If there's an open source algo then spammers will know exactly which buttons to push in order to dominate the search results, in fact they might even be able to build a piece of software that automatically adjusts spam pages to suit the latest releases of the algo.
Second, we have to be careful here to distinguish between what's control and what's just popularity.
MS controls the desktop because compatibility issues force many existing users (especially businesses) to continue with Windows. Leaving Windows means having to re-buy software and re-train staff (or yourself) in using the new OS, as well as migrate any data and (with businesses) make sure the new OS works with the rest of a company's infrastructure.
Google doesn't control search because users could use any search engine they want, there's absolutely no penalty for using (for example) Yahoo instead of Google. You can switch from one to the other or somewhere completely different at no cost to yourself. It just so happens that most people choose to stick with Google, but there's nothing to keep them there in the way there is with proprietary OSes.