Welsh police deployed facial recognition tech with a 92% false positive rate, but they're sure it's fine

The false positive rate here is not 92%

If you had a cancer detector such that, if you tested 100 people who did not have had cancer, it says “NO CANCER” to 99 of them, and “CANCER” in error to 1 of them, you would call that a false positive rate of 1%.

And if when you tested 100 people who did have cancer and it said “CANCER” to 99 of them and “NO CANCER” to 1 of them, you would call that a false negative rate of 1%.

If you then tested a population of 10000 people, 9900 of which did not have cancer and 100 of which did, you would expect there to be 99 false positives (1% of 9900) and 99 real positives (99% of 100).

What you have done is the equivalent of taking these stats - the output of a system with a 1% false positive rate and 1% false negative rate - and saying “Oh, there are 99 false positives and 99 real positives, so the false positive rate is 50%”

This is a statistical error so stupid and so common it has its own special name:

14 Likes