Forcing digital forensics to obey 'one size fits all' crime lab standard is 'stupid and expensive'
Prof hits out at looming regulations
Analysis Opposition is growing over demands that digital forensics labs comply with ISO 17025 – an international checklist for laboratory testing.
That means IT experts helping to nail murderers and miscreants for the plod must follow the same regulations that DNA labs and similar boffinry outfits obey – an approach critics argue will raise costs without improving results. The government's forensic science regulator has dismissed these concerns.
A recent survey [PDF] among digital forensic practitioners found that understanding about ISO 17025 is patchy. In addition, many respondents complained about the cost of implementing the standard, which is described as "general requirements for the competence of testing and calibration laboratories."
Patchy understanding – and even lower implementation – of new regs among digital forensics practitioners (Source: Digital Forensics Survey by Pat Beardmore, Geoff Fellows, Peter Sommer and others)
"ISO 17025 is regarded as both inappropriate – even useless – and expensive" for digital forensics, according to Peter Sommer, professor of digital forensics at Birmingham City University.
ISO 17025 sets the bar for high quality forensic science work, whether undertaken by the police or outside contractors. It's followed by labs tasked with matching DNA, fingerprints, blood, paint, fibre, and so on, ensuring that the test procedures and equipment are fair and valid.
Sommer argues that digital forensics is incompatible with the ISO standard, rendering the "one size fits all approach" advocated by the government inappropriate. He told The Register:
First, a typical examination of a PC or phone is not a single test operation (is there a match, or what percentage confidence do we have that there is a match?), but an investigation into a whole scene of crime where extant files, emails, photos, reconstruction of web browsing and social media activity – plus a review of hidden operating system files and logs – are all brought together.
Second, the speed of change in technology means that the relatively slow process of formal validation that applies in conventional forensics cannot be used in digital forensics, unless one recognises that digital forensic evidence will never be able to cope with devices that use recent operating systems and applications.
Third, the process of getting ISO 17025 certified is expensive both in preparation and in fees to the certifying body, UKAS [United Kingdom Accreditation Service].
Much digital forensics activity is concerned with reconstructing events and providing expert interpretations of data, rather than basic binary tests, and this makes ISO 17025 unsuitable for probing computers and devices, according to Sommer. He argued that matching DNA, fingerprints, paint fragments and fibres – the work of the mainstream forensics labs – is not exactly comparable with computer forensics, so a different approach is required.
Sommer told El Reg: "A number of established private sector digital forensic companies say that they will withdraw from police and publicly funded work" if the ISO 17025 is foisted upon them.
Cost vs benefits
Variable costs for ISO 17025 accreditation among computer labs (Source: Digital Forensics Survey by Pat Beardmore, Geoff Fellows, Peter Sommer and others)
A spokesperson for the UK's forensics regulator defended the looming requirements, arguing that ISO 17025 compliance offers tangible benefits, not least in upholding standards. In a statement, we were told:
The regulator has carefully considered all of the survey responses.
The ISO 17025 lab standard was chosen after extensive consultation with specialists in the field, and those practitioners who have already gained this accreditation are seeing real benefits.
Achieving this standard is crucial, as the Regulator has previously identified examples of quality failures which could have been avoided.
We are continuing to engage with the industry, and the Regulator has run a number of different programmes with both practitioners and the NPCC to help them achieve the new standards.
Although the regulator and the Home Office are constantly reviewing standards, El Reg understands there are no plans to change the current requirements nor the implementation timetable.
Sommer expressed disappointment at this line. He wants ISO 17025 compliance limited to certification for compliance to the initial evidence preservation stage and the development of existing good practice guides specific to computer forensics rather than adopting a new non-industry-specific framework.
"I fear that the regulator's continued rigid support for a costly and inappropriate standard will lower the quality of digital forensic evidence available to the criminal justice system, not improve it," Sommer told El Reg. "Scarce police resources will get diverted towards compliance bureaucracy as opposed to front line investigation."
"As fees for digital forensics work in the public sector are being reduced, a number of companies are withdrawing from that market," he concluded, adding that civil litigation and general cyber security work can be more lucrative than computer forensics work tied to assisting the courts and related to criminal prosecutions or defences. ®