Read two biometrics, get worse results - how it works
Iris algorithm originator shows how obvious isn't so obvious
A regular correspondent (thanks, you know who you are) points us to some calculations by John Daugman, originator of the Daugman algorithms for iris recognition. These ought to provide disturbing reading for Home Office Ministers who casually claim that by using multiple biometrics you'll get a better result than by using just the one. Although that may seem logical, it turns out that it it isn't, necessarily.
Daugman presents the two rival intuitions, then does the maths. On the one hand, a combination of different tests should improve performance, because more information is better than less information. But on the other, the combination of a strong test with a weak test to an extent averages the result, so the result should be less reliable than if one were relying solely on the strong test. (If Tony McNulty happens to be with us, we suggest he fetches the ice pack now.)
"The key to resolving the apparent paradox," writes Daugman, "is that when two tests are combined, one of the resulting error rates (False Accept or False Reject rate) becomes better than that of the stronger of the two tests, while the other error rate becomes worse even than that of the weaker of the tests. If the two biometric tests differ significantly in their power, and each operates at its own cross-over point, then combining them gives significantly worse performance than relying solely on the stronger biometric.
This is of particular relevance to the Home Office's current case for use of multiple biometrics, because its argument is based on the use of three types of biometric, fingerprint, facial and iris, which are substantially different in power.
Daugman produces the calculations governing the use of two hypothetical biometrics, one with both false accept and false reject rates of one in 100, and the second with the two rates at one in 1,000. On its own, biometric one would produce 2,000 errors in 100,000 tests, while biometric two would produce 200. You can treat the use of two biometrics in one of two ways - the subject must be required to pass both (the 'AND' rule) or the subject need only pass one (the 'OR' rule). Daugman finds that under either rule there would be 1,100 errors, i.e. 5.5 times more errors than if the stronger test were used alone.
He concludes that a stronger biometric is therefore better used alone than in combination, but only when both are operating at their crossover points. If the false accept rate (when using the 'OR' rule) or the false reject rate (when using the 'AND' rule) is brought down sufficiently (to "smaller than twice the crossover error rate of the stronger test", says Daugman) then use of two can improve results. If we recklessly attempt to put a non-mathemetical gloss on that, we could think of the subject having to pass two tests (in the case of the 'AND') rule of, say, facial and iris. Dropping the false reject rate of the facial test (i.e. letting more people through) in line with Daugman's calculations would produce a better result than using iris alone, but if the facial system rejects fewer people wrongly, then it will presumably be accepting more people wrongly.
Which suggests to us that simply regarding a second or third biometric as a fall back to be used only if earlier tests fail constructs a scenario where the combined results will be worse than use of the single stronger test, because in such cases the primary biometric test would have to be sufficiently strong to stand on its own, because you won't always be using the second or third test.
The deployment of biometric testing equipment in the field is also likely to have a confusing effect on relative error rates, because environmental factors will tend to impact the different tests to different degrees. Poor lighting may have an effect on iris and facial but not on fingerprint, while the aircon breaking down may produce greasy fingers and puffy red faces, but leave iris intact. Which would presumably mess up attempts to sync error rates.
But we feel ourselves beginning to intuit, and had perhaps best back off before phalanxes of irate mathematicians come after us. On the upside for the Home Office, Daugman points out that the combination of two tests of equal power - the iris patterns of both eyes, or two of a person's fingerprints - can enhance performance fairly easily. This actually provides some justification for the Home Office starting to count eyes and fingers individually, although the way they're putting it still sounds like the techies told them something, and now they're trying to repeat it without really understanding.
The extent to which they really do count the biometrics separately will also be important. Daugman points out that his calculations only deal deal with "decision-level fusion" (i.e. applying the decision rules to the individual biometrics separately), but there are other approaches such as sensor fusion, where the data is combined before decision rules are applied, or combining similarity scores before applying decision rules. As far as fingerprint is concerned, the Home Office certainly intends to have all ten prints on file, but there are all sorts of different ways that a test could read the data. Is a 'handslap' reading five individual biometrics read at once, or just the one? It depends how you treat it and how you use the decision rules on the data, and how you do this will have an effect on the validity of your claims about multiple biometrics. ®