"I have nothing to hide" - or the Sainsbury's Lesson

Guilt by association

The same patterns show that we have choices, sure, in our purchasing life...but that, in fact, most of us do follow the same paths others do.

So Sainsbury's was able to discard 95 per cent of the potential households in its area and say: "They aren't going that way. They might buy boot polish, but it's so unlikely as to be a complete waste of money putting a Kiwi voucher through the door."

On the other hand, the patterns might show quite clearly that 5,000 people bought something called "product A" and "product B" and "product C" and went on to buy "product D" three weeks later. So if you can dig out all the people who bought A, B and C but have not yet bought D, and mail them, you're clearly looking at a common pattern. So drop them a discount voucher for D!

And D turns out to be baby clothes, and the man of the household is vastly amused: "Darling, look at this! Those idiots at Sainsbury's are sending us baby clothes vouchers! As if we would want to start a family!" - and she says, turning pink and coughing: "Dear, I've been meaning to tell you, I've missed my last period..."

And this actually happened. As you can imagine, the angry letters poured in: "Who told you...!?" and the project was subtly modified.

That's how spooks work. They don't care what you're talking about until they know you're a threat. They work out whether you're a threat or not by looking for patterns (and in deference to my friends in Cheltenham, I won't go into much more detail) in your ordinary, everyday behaviour, which betrays who your friends are. Most of that information is stuff you'd never think to make secret.

And this is the important part - you'd never think to ask Parliament to say: "Protect me from this!"

Now, your credit card records are commercially confidential, but are they a State-protected secret? Your electricity bill payments, your bank statements, your phone call bills, your email address book...your Oyster card details for the subway rail, your fuel purchases...is any of that prohibited for Government databases?

Why would you care?

Now, look at the success of Touching the Void. It languished on the remainder list for two years until Amazon started telling people who read a different book on mountain climbing: "We see that people who read what you're reading, enjoyed this!"

Or look at the way Tivo works. It matches the sort of TV you actually watch with what you record, and actually anticipates your interest in new series' which you haven't even heard of yet, based on viewing patterns which it recognises.

Where am I going with this?

Imagine that you have an unfortunate trend in your thinking. Imagine that you find yourself following a train of thought which matches a sequence which has been seen before among enemies of the Government.

"Great!" you say. "This means that the enforcement people can actually anticipate terrorist acts? Spot potential terrorists before they even realise they're on the path to mayhem?"

Sponsored: Designing and building an open ITOA architecture