Original URL: http://www.theregister.co.uk/2012/07/30/wimbledon_ibm_sentiment_analysis/

Military-grade IBM kit senses love, hate in Wimbo fans' tweets

'Sentiment analysis' served at high speed

By Gavin Clarke

Posted in Servers, 30th July 2012 09:13 GMT

Although Andy Murray lost to Roger Federer at Wimbledon this month, he managed to ace the pair's Twitter contest.

Britain’s best tennis hope since Fred Perry didn't just feature in more tweets than his Swiss rival, he also surfed a tidal wave of plaudits on the social networking site.

More than 490,000 tweets mentioned Murray, of which 42 per cent were positive, compared to Federer’s 487,000, of which just 29 per cent were favourable.

The data comes courtesy of IBM, which for just over two decades has run the IT behind Wimbledon. This year branched out into a new area: analysis of watching fans’ “sentiment” by analysing tweets.

IBM inside Wimbledon 1, photo: Gavin Clarke

Back of the net: IBM's Wimbledon team monitor the website and SlamTracker on-site

IBM harvested tweets between 20 June and 11 July, and waded through 1.3 million messages from the day of the Murray-Federer final alone. The result: 1GB of data in an IBM DB2 database. That isn't a lot by web standards, or even USB or hard drive standards, but it was data that Big Blue considered important enough to bring in the big guns – IBM Content Analytics (ICA) with Enterprise Search, deployed at Wimbledon for the first time.

The US Army is among ICA's users and used the tech in a massive documents management overhaul.

Alan Flack, IBM’s programme executive for Wimbledon, told The Reg the sentiment analysis test with the military-grade ICA was a "learning exercise". Other Big Blue teams working for sports events backed by the company's kit will be able to build on knowledge gained at Wimbledon this year.

“We will share some insights with the client, and amongst the IBM team, but this test was really aimed at building something that could be used for other organisations and brands,” he said. “Feedback about the event and the players could both be useful and interesting."

IBM provides analytics for other sports: for example, it announced the Leicester Tigers rugby club has started using the company's SPSS Modeller to predict players' performance and spot potential injuries.

The computer giant became the official IT supplier to the All England Lawn Tennis Club and Wimbledon Championships in 1990. Since then the competition has migrated to IBM’s servers, software, code and its cloud.

A very grand Slam

One of the most visible applications is IBM SlamTracker, a system that dishes out match information to the media and watching public. The facts and figures are served to display boards, the Wimbledon website and its native iOS and Android smartphone apps.

SlamTracker holds Grand Slam tennis records dating back seven years - 39 million data points in total - on players and matches. The information is recalled using IBM’s SPSS predictive analytics software. Data points are fed into SlamTracker using a combination of court-side humans tapping on a specially built tennis app and electronic monitoring.

In 2011 IBM began testing SecondSight on court 18. This technology captures players’ movements, tracks their position and can recreate it in 3D on-screen. SecondSight is for players, coaches, commentators and fans, and was this year rolled out to the Centre Court - where the Murray-Federer final was played.

Data for SlamTracker and SecondSight sits in an IBM DB2 database running on Linux that's part of the Wimbledon Information System, which is on-site in the grounds of the All England Lawn Tennis Club.

Rapid rollout while avoiding a double fault

It's an indication of the perceived value of this data that it took just two tournaments for SecondSite to be rolled out to the world-famous court. Compare this rapid expansion to the decades of debate over whether or not the Centre Court should have a roof; arguments raged over the effect this cover may have on the game and that rain-hit postponed matches were part of the Wimbledon experience.

IBM, meanwhile, this year tore the wraps off a new Wimbledon.com website that brought live video to online spectators in addition to player and match stats. The website is hosted by three IBM data centres in the US, as part of IBM’s SmartCloud to scale according to load. The Wimbledon site can receive up to 450 million hits in the fortnight of the tournament. Big Blue has been running the Wimbledon web servers on Linux since 2004.

This all sounds very nice and webby and mobile, but what about the hard data from those tweets?

IBM inside Wimbledon 2, photo: Gavin Clarke

IBMers watch Big Blue's cloud to ensure Wimbledon.com avoids a fault

Outside the US Army, ICA has a track record in financial services, medical insurance and local government. It's also used by the UK's Department of Work and Pensions. ICA searches structured and unstructured data using natural language processing to understand the meaning and context of language and words used.

When it came to Wimbledon, IBM didn’t just rely on the tweets and their accompanying metadata (such when a message was tweeted and by whom) to understand what was being said: IBM also generated an additional piece of metadata called a “sentiment score”, a value ranging from minus-five to plus-five. ICA created an additional index for tweets to help the software perform language-based queries such as “show me tweets containing the word 'Wimbledon’” and sort tweets by the frequency of adjectives used.

ICA queries were built using a concept of facets. Verbs, nouns and adjectives were one type of facet while IBM defined other types related specifically to tennis, such as players' names and tennis terminology including deuce, service and net. This helped identify trends and form queries specific to the competition.

Among other stats guzzled from Twitter's fire hose of messages: former world number-one champion Rafael Nadal, who lost in an early round surprise shocker to Lukas Rosol, was on the receiving end of the most negative tweets of all the top seeds – 20 per cent negative compared to 15 per cent positive. Rosol scored more positive tweets than Nadal, 28 per cent, but scored fewer mentions - 64,000 to 143,864.

The sportsperson as 'brand'

Wimbledon is a showpiece for IBM. New, existing and prospective customers are routinely given a tour of the facilities the tech giant runs at the tennis grounds. What people see is a bunker of IBM staff with servers and PCs monitoring data feeds, and server and website performance, all while keeping the website and analytics software running for SlamTracker, SecondSight, Wimbledon.com and the cloud.

Sentiment analysis may seem wooly and vague now, but Flack believes it’s got potential for any brand – not just those in sports – to find out what people are saying about them, gauge wants and needs, and to understand and react to opinions. ®