Drowning in data - complexity's threat to terror investigations
Knowing everything, except what happened?
Analysis A Home Affairs Committee report into police detention powers, published earlier this week, concludes that police powers to hold terror suspects without charge will need to be extended from 28 days to 90 days - and, once the flimsier justifications (e.g. time needed for prayers) have been stripped out, technology is largely to blame. The Committee, which has an impressive track record of criticising the Government but somehow ending up agreeing with it anyway, takes into account the international nature of current terrorist threats, the security services' need to mount 'pre-emptive' operations in order to 'protect the public', encryption, the burden of data analysis, and the logistics of forensics in general in order to come to its conclusions.
But under all that it's thin stuff, and although the Committee takes swipes at both the Government and the Association of Chief Police Officers (ACPO), it bases its own conclusions on pretty much the same absence of evidence previously deployed by the police, the late Charles Clarke and Tony Blair himself. There is a problem there, and technology does have a lot to do with it, but the Committee report misses its nature just as widely as the rest of them have.
What we've got Before we crack on, we'll just do a swift reality check of the current situation vis a vis terror investigations. Despite an extremely messy Parliamentary argument (one of several) last Autumn which resulted in police detention powers being extended from 14 days to 28 days, the Home Office has yet to switch the 28 days on, and the world has not yet ended. The matter was apparently sufficiently urgent for Tony Blair to have his staff whack out a two-week long "consultation" on the back of a fag-packet last August, prior to the Dear Leader and family sponging a holiday off... who was it, Sir Cliff? Yes, we recall it was. But it wasn't sufficiently urgent for the subsequent legislation to have been actually implemented. Home Office figures published in the report also do not indicate a pressing need for lengthy periods of pre-trial detention, with the vast majority of arrests being dealt with within the previous period of seven days (it was extended to 14 in January 2004), and only 11 people in total held for as long as 13-14 days during 2004-5. But, as the Home Affairs Committee's evidence-based report has it, you never know...
Similarly, Part III of the Regulation of Investigatory Powers Act 2000 provides for a two year prison term for failure to disclose an encryption key, but although all of the usual Government and security suspects (inc., the Committee) think this should be brought into force, it hasn't been. The aforementioned suspects seem equally agreed that it wouldn't do a lot of good anyway, on the basis that your thinking terrorist is going to keep schtum and do the two years rather than a lengthier terror stretch, but they'd still all like it brought in. Charles Clarke incidentally explained to the Committee that doing this hadn't been urgent after all because the amount of encrypted data police had encountered had turned out to be less than had been expected. But apparently we need it anyway because, well, you never know... Again.*
'It's inevitable...' The Committee's report accepts that the increasing number of investigations, together with their increasing complexity, will make longer detention inevitable in the future. The core calculation is essentially the one put forward by the police and accepted by the Government - technology has been an enabler for international terrorism, with email, the Internet and mobile telephony producing wide, diffuse, international networks. The data on hard drives and mobile phones needs to be examined, contacts need to be investigated and their data examined, and in the case of an incident, vast amounts of CCTV records need to be gone through. As more and more of this needs to be done, the time taken to do it will obviously climb, and as it's 'necessary' to detain the new breed of terrorist early in the investigation before he can strike, more time will be needed between arrest and charge in order to build a case.
All of which is, as far as it goes, logical. But take it a little further and the inherent futility of the route becomes apparent - ultimately, probably quite soon, the volume of data overwhelms the investigators and infinite time is needed to analyse all of it. And the less developed the plot is at the time the suspects are pulled in, the greater the number of possible outcomes (things they 'might' be planning) that will need to be chased-up. Short of the tech industry making the breakthrough into machine intelligence that will effectively do the analysis for them (which is a breakthrough the snake-oil salesmen suggest, and dopes in Government believe, has been achieved already), the approach itself is doomed. Essentially, as far as data is concerned police try to 'collar the lot' and then through analysis, attempt to build the most complete picture of a case that is possible. Use of initiative, experience and acting on probabilities will tend to be pressured out of such systems, and as the data volumes grow the result will tend to be teams of disempowered machine minders chained to a system that has ground to a halt. This effect is manifesting itself visibly across UK Government systems in general, we humbly submit. But how long will it take them to figure this out?
It's fairly easy to see how one facet of the problem, volume of cases, grows like topsy. The Forest Gate raid is by no means the only case where resource-intensive raids and arrests have been based on doubtful tips and flimsy evidence, and while for reasons of sub judice we can't go into many of these cases in any great depth, published data on the charges that have been brought is surely significant. Few terrorism arrests lead to terrorism charges, and in the case of 'Islamist' category arrests, the charges ultimately brought are often immigration, credit card or ID fraud related. People are pulled in because the security forces believe they 'might' be terrorists, 'might' be about to launch a huge chemical, biological, nuclear attack, 'might' be suicide bombers.
As indeed they might be, although the more level-headed among us might wish for a better grasp and deployment of probability and risk assessment on the part of the security services. And indeed, for a more realistic approach from Government. The trend here in legislation has been to inexorably broaden the range of criminal offences (in general, but particularly so with reference to terrorism), the result being that a wide range of things that are commonly (or at least easily) done can be deemed crimes. Depending. Possession of the Encyclopaedia of Jihad, for example, was used in last year's prosecution of Abu Hamza, Monster of Finsbury Park Mosque. In the references in his most excellent book on al-Qaeda, though, Observer hack Jason Burke refers to a copy of the very same work being "in the author's possession." Burke however remains mysteriously at liberty. Similarly, researching fusing and detonators is in some contexts a dead giveaway (charges have been brought in several cases in the UK), but this week in particular, several US web sites have been happily and innocently explaining fusing and detonators in order to help people build impressive 4th July rockets.
* One gotcha of the arguments (bizarrely, they can all agree but still have arguments) is that the more publicity the issue gets, the more likely terror groups are to use encryption. Few do at the moment, and security awareness among Islamist groups (even the allegedly experienced ones) is frequently low. Another gotcha arises as and when encryption is widely used. If it's poor and badly set up, then it's easy to crack and you don't need the key. If it's properly set up, as Professor Ross Anderson put it to the Committee you either guess the password or give up. No amount of analysis time will have any bearing on this, and as far as the encryption issue goes, 90 days is neither here nor there.
Obviously, the context of a suspect's actions is important in an investigation, whether or not those actions are in themselves illegal. If for example it had been known that the future July bombers were researching detonators then concern and further investigation would have been in order. The question of whether or not an action is illegal is however important. A legal but suspicious action requires investigation of context in order to determine intent, and to identify the actual crime, whereas an illegal action (which nevertheless might have a perfectly innocent explanation) allows prosecution without reference to or investigation of context.
And the wrong people sometimes get sent down. That however is not the immediate problem from the point of view of the system. Widespread prosecution for trivial and low-level offences will tend to overload the system and reduce focus on potentially more serious offences, while choking processes with low-level and irrelevant data, and directing resources down blind alleys. Police will frequently find themselves failing to find the conspiracy in cases where there really wasn't one.
Whatever though, once a suspect is on the radar they get turned over and their house torn apart, at which point evidence may emerge proving they're a more prosaic class of criminal instead (Or as well as...? Well, they might be). Alternatively, their computer data might reveal visits to suspicious sites or shady chatrooms, possibly meaning they're in possession of information likely to prove useful to terrorists, and possibly also producing other names, possibly in other countries, of people whose homes can be dismantled too. This also works in the other direction - several recent arrests in the UK suggest that police have been following up overseas tips based on the monitoring of email contacts and/or web site visits. Thus do the numbers grow.
It is, as the police would have it, perfectly possible that those charged with non-terrorist or low-level offences would later have committed more serious acts, and/or that longer and more intensive investigation would have uncovered the serious acts they intended to commit, and the mysterious terror mastermind. But it is also perfectly possible, and given the number of 'suspects' considerably more likely, that they weren't real terrorists in the first place and you've just bust the budget trying to prove they were.
How many? A few months ago press reports of the number of terror suspects in the UK climbed steadily to 1,200, a figure from theDaily Telegraph which is quoted in the Home Affairs Committee report. The Independent, however speaks of MI5 now investigating 8,000 (a BBC report puts the number as "less than one per cent" of a total of approximately 1.5 million UK muslims) al-Qaeda "sympathisers" , i.e. people who might become terrorists. As indeed they might do, but the number you come up with is always going to depend on how widely you cast the 'might' net. Back in the 70s and 80s there were quite possibly more than 8,000 sympathisers for the Red Brigades or the Red Army Fraction in the UK, and most certainly well over 8,000 for the Provisional IRA. The move from sympathiser to activist was however too great for most of them, and the technology of the time did not support the widespread monitoring of, for example, unguarded conversations in North London pubs. Today's technology and today's and tomorrow's legislative environment do however support this, meaning that the security services have an impressive and exponentially growing number of unguarded electronic pub conversations to follow up.
Most of which will be rubbish, kids who're never going to do anything, talking big. But some of them won't be. Even then though, the would-be terrorists aren't necessarily as dangerous as the headlines (themselves often orchestrated by security service "sources") would have us believe. The much-hyped and hugely expensive ricin affair (see Clarke calls for ID cards after imagining huge poison terror ring) was never any kind of threat, while the apparently non-existent chemical weapon police were seeking at Forest Gate could not have been built in any effective configuration by non-state organisations; anything that could currently be built by terrorists almost certainly would not work, or at best (from their point of view, obviously), would have been gloriously ineffective (see Homebrew chemical terror bombs, hype or horror?).
There we have two UK instances of large quantities of investigative resources being expended on first, a nutter (who nevertheless clearly should have been picked up), and on a threat that manifestly could not have existed. The Met, incidentally, strove long and hard to establish an international 'al-Qaeda connection' to the nutter, claiming last October that the investigation spanned "26 other jurisdictions" in addition to the UK. It said of this case: "The challenge was to analyse a huge amount of material, to identify the prime conspirators (and what it was they were plotting to do), and to clarify the roles played by each of the suspects. This proved impossible in the time available..." Indeed - but even if the conspiracy was more than the product of fevered imaginations triggered by a squealer in Algerian custody, the fact remains that it was centred on a joke recipe for a relatively ineffective poison. Would resources not have been better deployed against more immediately dangerous threats?
The State they're in Forest Gate shows that since the ricin affair the security services have acquired little scientific knowledge of the substances they're in hot pursuit of, and remain fixated on the largely illusory threat of chemical, biological and nuclear attacks. This clearly leads to pre-emptive actions that need not have happened.
The Met document describing the "challenge" of the ricin affair, a letter from the Met's Andy Hayman to then Home Secretary Charles Clarke in support of 90 day powers, also includes a "Theoretical Case Study", which is part bad movie terror threat script, part the Met's dream mega terror case. A "reliable" tip tells of terror attacks planned on the Houses of Parliament and British Embassies in Pakistan, Istanbul and "Morrocco" (sic). The attacks will take place in three months, using conventional and homemade explosives, and possibly CBRN (Chemical, Biological, Radiological and Nuclear). The tale then describes surveillance, 15 arrests, 55 forensic searches throughout the country, 4,000 exhibits, 600 documents in Arabic, boxes of Arabic videos, 100 ID documents, "over 268" seized computers, 274 hard disks, 591 floppy disks, 920 CD DVDs, 47 zip drives, 60 mobile phones, 25,000 man hours spent on CCTV, 3,674 analysing eavesdropping material, evidence gathering in 17 countries... The list goes on and, as the Met letter says ("statistics used are entirely typical"), it's a fair estimation of what the security services do when they think they're on to something.
That does not however make what they do sensible. There was some dispute during the Home Affairs Committee sessions over how long it took to image a hard disk, but whether its 30 minutes or 12 hours (which is what it is in Met procedure) isn't massively relevant. It's the time that's spent poring over the data on the hard disk, searching everywhere for the slightest clue, that's the killer. Police are to some extent now being selective in what they try to analyse, but it is probably still more a case of them deciding whether or not to conduct an extensive analysis of a whole hard disk, rather than being selective within it (e.g. a check of the email client, browser cache and other likely places). And with reference to mobile phones, witness Vinesh Parmar of the digital crime unit of LGC Ltd made a telling point: "Too often we get requests which say we want everything, which in reality is not a workable request. What we find is that law enforcement agencies need to start understanding the data that is available and to start understanding what is possible evidence or what is intelligence... At the moment a lot of work we do is fishing expeditions where we are basically requested to grab everything out of there and we do not know the case history."
To some extent this is the product of the completist police mentality that demands that all computers with the slightest connection to an enquiry be seized and examined in painstaking detail, but it's also what you'd expect to happen if police had first arrested a suspect, and second started to try to find the crime.
Just imagine.. In Hayman's showcase exhibit the threat is imaginary, but one suspects imagination can also form a component of the real thing. Real terror cases and claimed terror plots frequently include plans to attack major public buildings, tall buildings (e.g. Canary Wharf), international airports, and references to CBRN weapons use. Few if any of those that have been "frustrated" or documented so far include convincing plans (even plans, full stop) for actually mounting the attacks, sourcing the deadly poisons and constructing the weapons. Transcripts meanwhile are peppered with lurid and unfeasible attack ideas (often sounding uncannily like the sort of thing a mouthy teenager would say to impress his mates) and references to 'terror manuals' which often turn out to be dodgy survivalist poison recipes and/or the ubiquitous Encyclopaedia of Jihad which, as it includes references to tall buildings, is a handy fall-back if the prosecution is in want of a target list.
We can, by going a little theoretical ourselves, try to understand what's happening in such cases. Young men disaffected with the state (you know why) get together, talk, consider actions. Being young they're Internet and mobile phone aware, so they use technology for some of their communications and maybe contact similarly disaffected young men in other countries. They consider bombs, and they've heard a hell of a lot about tall buildings, aircraft, chemicals and poisons so guess what, they talk about these too, and maybe they start researching how to do it. And guess what? They're almost certainly going to find those very same dodgy poison recipes, excerpts from the Encyclopaedia of Jihad and a few beheading videos. Are they dangerous yet? Are they a major international terror plot to be frustrated? Probably not. Yet. But it's the first anniversary of the July bombings this week, and that's reason enough to accept that disaffected youth can grow into real terror.
There is clearly a major problem for the security services in distinguishing disaffected talk from serious planning, and in deciding when an identified group constitutes a real threat. But the current technology-heavy approach to the threat doesn't make a great deal of sense, because it produces very large numbers of suspects who are not and never will be a serious threat. Quantities of these suspects will nevertheless be found to be guilty of something, and along the way large amounts of investigative resource will have been expended to no useful purpose, aside from filling up 90 days. Overreaction to suggestions of CBRN threats is similarly counter-productive, because it makes it more likely that nascent groups will, just like the police, misunderstand the capabilities of the weapons, and start trying to research and build them. Mischaracterising the threat by inflating early, inexpert efforts as 'major plots' meanwhile fosters a climate of fear and ultimately undermines public confidence in the security services.
The oft-used construct, "the public would never forgive us if..." is a cop-out. It's a spurious justification for taking the 'collar the lot' approach, throwing resources at it, ducking out of responsibility and failing to manage. Getting back to basics, taking ownership and telling the public the truth is more honest, and has some merit. A serious terror attack needs intent, attainable target and capability, the latter being the hard bit amateurs have trouble achieving without getting spotted along the way. Buying large bags of fertiliser if you're not known to the vendor and you don't look in the slightest bit like a farmer is going to put you onto MI5's radar, and despite what it says on a lot of web sites, making your own explosives if you don't know what you're doing is a good way of blowing yourself up before you intended to. If disaffected youth had a more serious grasp of these realities, and had heard considerably more sense about the practicalities, then it's quite possible that fewer of them would persist with their terror studies. Similarly, if the general public had better knowledge it would be better placed to spot signs of bomb factories. Bleached hair, dead plants, large numbers of peroxide containers? It could surely have been obvious.
Does that work? Does it get us very far? No, in the sense that it doesn't stop the sympathisers from sympathising and it doesn't stop all of the bombs. But given that neither of these is going to happen whatever the police do, and whatever the law says, we need a long-term survival/endurance strategy that doesn't drown the security services in a swamp of data, doesn't turn us into a police state, but does whatever is feasible to minimise risk. Despite what they (inc., the Home Affairs Committee) tell you, we've been here before, and it isn't all that different this time around. ®