Feeds

"I have nothing to hide" - or the Sainsbury's Lesson

Guilt by association

Choosing a cloud hosting partner with confidence

Comment How frightened would you be if you were secretly planning to get pregnant, without telling your husband, and discovered that someone had written to him telling him about it? Or, put the other way, how would you feel if you discovered your wife was pregnant only when someone dropped you a letter?

And who would that person be? It would have to be someone like Robbie "Cracker" Coltrane, right? A deep profiler with the power of a hypnotist...? Or it would have to be a Government spook. They could do it, surely, the way the world's spooks monitor all of us: easy. They tap our phones, right?

Well, the thing to remember is that it really did happen. And no, if you thought it was phone tapping, you're wrong, and it's a misunderstanding which I won't make any friends for clearing up. But you need to understand the basic principles of data mining to understand why the world of spooks and the world of search engines are about to overlap, and why you should be nervous about this.

The lesson here is one I call "The Sainsbury's Lesson" when doing presentations for technical audiences, because I was taught this by a data miner who worked for the giant British supermarket of that name.

The story, summarised, is that Sainsbury's was spending an absurd amount of money sending people promotional coupons, money-off special offers, and other junk mail to encourage them to swing by the Sainsbury's supermarket next time, rather than Waitrose or Safeway or Asda - and it was pretty hard to be sure it was actually doing any good.

The trouble was simple: they were sending girly shampoo promotions to households with six rugby-playing male students, or home improvement promotions to households with one elderly pensioner with osteoporosis, or bulk beer deals to households where they were all strictly teetotal. Not profitable stuff. And their IT staff heard about this and said: "But you don't have to do that!"

This goes back a bit before the days of Nectar, when Sainsbury's had its own loyalty card, plus it sold fuel out of its own petrol stations and ran a bank and a credit card. And the IT people said: "If you know what sort of things someone buys you can make a pretty good guess about what they may want to buy next."

The beauty of the system was that data mining requires no intellectual engagement. The Sainsbury's Lesson could be called the Amazon Lesson, or the Tivo Lesson for that matter. All you do is look for patterns. The more frequently you find a pattern, the better it is to guide you.

Take a commuter travelling from London to Reading. They can get into a car and drive. They can get into a taxi and be driven. They can take a bus. But most will get onto a train, with a choice of catching the Great Western service from Paddington, or the slower SouthWest service from Waterloo. If you want to catch them before they get to Reading, you aren't going to do it by chance.

But suppose you get information that says that they have passed through, Ascot, Sunningdale, Virginia Water, and Egham? If you know the route, you know that the next station, going west, will be Staines. Now, it's perfectly possible that they aren't going to be at Staines next. They might get off the train at Egham, take a cab to the station beyond Staines, and rejoin the train there... but seriously, what are the chances of that?

Security for virtualized datacentres

More from The Register

next story
Ex-US Navy fighter pilot MIT prof: Drones beat humans - I should know
'Missy' Cummings on UAVs, smartcars and dying from boredom
Facebook, Apple: LADIES! Why not FREEZE your EGGS? It's on the company!
No biological clockwatching when you work in Silicon Valley
The 'fun-nification' of computer education – good idea?
Compulsory code schools, luvvies love it, but what about Maths and Physics?
Doctor Who's Flatline: Cool monsters, yes, but utterly limp subplots
We know what the Doctor does, stop going on about it already
'Cowardly, venomous trolls' threatened with TWO-YEAR sentences for menacing posts
UK government: 'Taking a stand against a baying cyber-mob'
Happiness economics is bollocks. Oh, UK.gov just adopted it? Er ...
Opportunity doesn't knock; it costs us instead
Sysadmin with EBOLA? Gartner's issued advice to debug your biz
Start hoarding cleaning supplies, analyst firm says, and assume your team will scatter
Don't bother telling people if you lose their data, say Euro bods
You read that right – with the proviso that it's encrypted
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.