Original URL: https://www.theregister.com/2013/01/09/twitter_human_computation/
Twitter's real time data cruncher is made of people
Crowdsourcing advocates will be in your ear about this forever
Posted in Networks, 9th January 2013 05:25 GMT
Twitter has revealed that real, live, flesh-and-blood people are one of its key tools for analysing data in real time, and that it finds the people it needs with an unconventional approach to crowdsourcing.
The micro-blog’s engineering team has published a post in which it explains that trending topics post a tricky challenge as it is rather hard to build search or advertising tools that understand “… that #bindersfullofwomen refers to politics, and not office accessories …”.
The post goes on to say “ … we need to teach our systems what these queries mean as quickly as we can — because in just a few hours, the search spike will be gone”. The best way to understand the new flood of searches, Twitter has come to appreciate, is by putting people to work.
As a Californian company Twitter obviously can’t help itself and has labelled its use of Amazon Web Services’ Mechanical Turk crowdsourcing service “a real-time human computation engine”. If you can get past that, the post explains that “As soon as we discover a new popular search query, we send it to our human evaluators, who are asked a variety of questions about the query.”
Those evaluators, or “judges” may be asked to “ … categorize the query, or provide other information (e.g., whether there are likely to be interesting pictures of the query, or whether the query is about a person or an event) that helps us serve relevant Tweets and ads.”
Much of the work to deliver work to judges is automated. The post says real time computation system Storm does a lot of the heavy lifting, directing traffic for Hadoop and other tools.
The post also reveals that the judges Twitter hires work just about full time for the company. Twitter draws workers from a custom pool of Mechanical Turks because the quality of random crowdsourced workers “… doesn’t always meet our needs.” The company instead works with folks it knows are “experts at the kinds of tasks we send, and can often provide higher quality at a faster rate than what even in-house judges provide.” The company has even created online communities for its judges and says they use them to “… collaborate amongst themselves to give us the best judgments…”.
All of which is something for the crowdsourcing crowd to crow about, even if the idea that a large-ish corporation valued at billions of dollars is effectively employing people without the responsibilities that come with formally hiring them will raise eyebrows elsewhere. Twitter points out the work on offer is flexible and that its team of judges “… can work anywhere, anytime — which is a requirement for this system, since global event spikes on Twitter are not limited to a standard 40-hour work week.” ®