Das überdatabase: Inside Wacky Jacqui's motherbrain

Analysis Home Secretary Jacqui Smith isn't known in these pages for the clarity of her pronouncements on technology. And yesterday, as she confirmed the government's plan to proceed with the Interception Modernisation Programme (IMP), she limited herself to the spin of building a universal communications surveillance apparatus.

The details of the accompanying Communications Data Bill will be opened to consultation in the new year, she said, with the aim of achieving consensus with "interested parties". Smith was keen to emphasise the content of every phone, internet and mobile communication will not be harvested, but the details of who contacts whom, when and where. That distinction is likely be the cornerstone of attempts to sell IMP to MPs and a public wearied by the erosion of civil liberties and major government data losses.

Says who?

Smith was clear that she won't take "no" for an answer. "All this is a reflection of the technological and behavioural changes that the growth of the internet brings. Once again, that is not a Government policy which is somehow optional. It is a reality to which Government needs to respond," she said.

The follow-up propaganda push has already begun today. In The Times an unnamed source, clearly with a strong desire to see IMP built, spoonfed a dubious and old story about the threat posed by Skype and other VoIP applications to counter-terror operations. The hungry Thunderer hacks swallowed the security services' line that "internet phone calls are crippling fight against terrorism". No quotation marks in that headline, no opposing view in the story: it's being crippled people - fact.

Then with impeccable deadpan delivery, we're told in the penultimate paragraph that "The Times has learnt that police chiefs are to begin a discreet lobbying exercise in favour of the new powers". No, really?

Such media glove-puppetry will intensify in the coming months. People with knowledge of IMP developments in Whitehall and industry told The Register over summer that Jacqui and the security services do not speak for the whole of the government on the project. One memorably said the intelligence services' suggestion that IMP would seek to standardise all UK communications data and record details of every chat session had been described as "science fiction" by ISPs. The Cabinet Office and Treasury are also known to have major reservations.

But why?

The reasons why GCHQ and others have been able to persuade ministers that spending £12bn - the budget figure being floated in government - is a good idea despite fierce opposition IMP has and will face demand better illumination. The project is best explained as two simultaneously parallel and intertwining strands which will allow snoopers to very quickly see who their targets communicate with, and - with a warrant - what they are saying:

• The überdatabase

  A massive central silo for collecting, processing and cross reference details of emails sent, websites visited, calls made, mobile phone cell locations and more. Probably administered at "the doughnut", GCHQ's iconic Cheltenham spy station. This data won't be intercepted, but ISPs and telcos will provide it in real time from their own back end systems.

• The "black boxes"

  This network of wiretaps will put the "interception" into "Interception Modernisation Programme", using the information in the central comunications data silo to target suspects' traffic. It'll be hugely expensive to ensure the complete coverage spy chiefs want, but the plan is that any internet protocol (IP) communication will be available for interception. It means calls, texts, gaming, chat and web browsing will all be laid bare to analysts.

Arrangements have long been in place with major telecoms and internet providers to access a lot of this type of data on a case-by-case basis. It's proved valuable as an investigative tool and as evidence in countless crimes; 95 per cent of serious crimes since 2004, the Home Secretary said yesterday. It's known that major communications firms have people on staff who are effectively employed by the intelligence services, ready to grab any data required.