Drowning in data - complexity's threat to terror investigations

Knowing everything, except what happened?

Analysis A Home Affairs Committee report into police detention powers, published earlier this week, concludes that police powers to hold terror suspects without charge will need to be extended from 28 days to 90 days - and, once the flimsier justifications (e.g. time needed for prayers) have been stripped out, technology is largely to blame. The Committee, which has an impressive track record of criticising the Government but somehow ending up agreeing with it anyway, takes into account the international nature of current terrorist threats, the security services' need to mount 'pre-emptive' operations in order to 'protect the public', encryption, the burden of data analysis, and the logistics of forensics in general in order to come to its conclusions.

But under all that it's thin stuff, and although the Committee takes swipes at both the Government and the Association of Chief Police Officers (ACPO), it bases its own conclusions on pretty much the same absence of evidence previously deployed by the police, the late Charles Clarke and Tony Blair himself. There is a problem there, and technology does have a lot to do with it, but the Committee report misses its nature just as widely as the rest of them have.

What we've got Before we crack on, we'll just do a swift reality check of the current situation vis a vis terror investigations. Despite an extremely messy Parliamentary argument (one of several) last Autumn which resulted in police detention powers being extended from 14 days to 28 days, the Home Office has yet to switch the 28 days on, and the world has not yet ended. The matter was apparently sufficiently urgent for Tony Blair to have his staff whack out a two-week long "consultation" on the back of a fag-packet last August, prior to the Dear Leader and family sponging a holiday off... who was it, Sir Cliff? Yes, we recall it was. But it wasn't sufficiently urgent for the subsequent legislation to have been actually implemented. Home Office figures published in the report also do not indicate a pressing need for lengthy periods of pre-trial detention, with the vast majority of arrests being dealt with within the previous period of seven days (it was extended to 14 in January 2004), and only 11 people in total held for as long as 13-14 days during 2004-5. But, as the Home Affairs Committee's evidence-based report has it, you never know...

Similarly, Part III of the Regulation of Investigatory Powers Act 2000 provides for a two year prison term for failure to disclose an encryption key, but although all of the usual Government and security suspects (inc., the Committee) think this should be brought into force, it hasn't been. The aforementioned suspects seem equally agreed that it wouldn't do a lot of good anyway, on the basis that your thinking terrorist is going to keep schtum and do the two years rather than a lengthier terror stretch, but they'd still all like it brought in. Charles Clarke incidentally explained to the Committee that doing this hadn't been urgent after all because the amount of encrypted data police had encountered had turned out to be less than had been expected. But apparently we need it anyway because, well, you never know... Again.*

'It's inevitable...' The Committee's report accepts that the increasing number of investigations, together with their increasing complexity, will make longer detention inevitable in the future. The core calculation is essentially the one put forward by the police and accepted by the Government - technology has been an enabler for international terrorism, with email, the Internet and mobile telephony producing wide, diffuse, international networks. The data on hard drives and mobile phones needs to be examined, contacts need to be investigated and their data examined, and in the case of an incident, vast amounts of CCTV records need to be gone through. As more and more of this needs to be done, the time taken to do it will obviously climb, and as it's 'necessary' to detain the new breed of terrorist early in the investigation before he can strike, more time will be needed between arrest and charge in order to build a case.

All of which is, as far as it goes, logical. But take it a little further and the inherent futility of the route becomes apparent - ultimately, probably quite soon, the volume of data overwhelms the investigators and infinite time is needed to analyse all of it. And the less developed the plot is at the time the suspects are pulled in, the greater the number of possible outcomes (things they 'might' be planning) that will need to be chased-up. Short of the tech industry making the breakthrough into machine intelligence that will effectively do the analysis for them (which is a breakthrough the snake-oil salesmen suggest, and dopes in Government believe, has been achieved already), the approach itself is doomed. Essentially, as far as data is concerned police try to 'collar the lot' and then through analysis, attempt to build the most complete picture of a case that is possible. Use of initiative, experience and acting on probabilities will tend to be pressured out of such systems, and as the data volumes grow the result will tend to be teams of disempowered machine minders chained to a system that has ground to a halt. This effect is manifesting itself visibly across UK Government systems in general, we humbly submit. But how long will it take them to figure this out?

It's fairly easy to see how one facet of the problem, volume of cases, grows like topsy. The Forest Gate raid is by no means the only case where resource-intensive raids and arrests have been based on doubtful tips and flimsy evidence, and while for reasons of sub judice we can't go into many of these cases in any great depth, published data on the charges that have been brought is surely significant. Few terrorism arrests lead to terrorism charges, and in the case of 'Islamist' category arrests, the charges ultimately brought are often immigration, credit card or ID fraud related. People are pulled in because the security forces believe they 'might' be terrorists, 'might' be about to launch a huge chemical, biological, nuclear attack, 'might' be suicide bombers.

As indeed they might be, although the more level-headed among us might wish for a better grasp and deployment of probability and risk assessment on the part of the security services. And indeed, for a more realistic approach from Government. The trend here in legislation has been to inexorably broaden the range of criminal offences (in general, but particularly so with reference to terrorism), the result being that a wide range of things that are commonly (or at least easily) done can be deemed crimes. Depending. Possession of the Encyclopaedia of Jihad, for example, was used in last year's prosecution of Abu Hamza, Monster of Finsbury Park Mosque. In the references in his most excellent book on al-Qaeda, though, Observer hack Jason Burke refers to a copy of the very same work being "in the author's possession." Burke however remains mysteriously at liberty. Similarly, researching fusing and detonators is in some contexts a dead giveaway (charges have been brought in several cases in the UK), but this week in particular, several US web sites have been happily and innocently explaining fusing and detonators in order to help people build impressive 4th July rockets.

* One gotcha of the arguments (bizarrely, they can all agree but still have arguments) is that the more publicity the issue gets, the more likely terror groups are to use encryption. Few do at the moment, and security awareness among Islamist groups (even the allegedly experienced ones) is frequently low. Another gotcha arises as and when encryption is widely used. If it's poor and badly set up, then it's easy to crack and you don't need the key. If it's properly set up, as Professor Ross Anderson put it to the Committee you either guess the password or give up. No amount of analysis time will have any bearing on this, and as far as the encryption issue goes, 90 days is neither here nor there.

Sponsored: 5 critical considerations for enterprise cloud backup