How facial recognition works (and how to hack your own in Python)

Originally published at: http://boingboing.net/2016/07/25/how-facial-recognition-works.html

1 Like

Facebook’s recognition software is 98% accurate? All I know is that it keeps tagging me as my father.

1 Like

I think I opted out of FB’s auto-recognition–or maybe just convinced myself that I did–but Google’s is frighteningly good. When I dumped a bunch of old photographs onto their hosting service, I found that it correctly matched my friends’ five-year-old son with a picture of his 18-month-old self. These were pictures taken with different cameras, with no GPS data or other input from me to help it “cheat.”

Then I spent about a month trying without much success to take pictures of people in ways that would fool it. Even blurry crappy scanned-in artifacted disposable camera pictures from the 90s work just fine.

I’m glad it happened. I’d fallen into the trap of thinking that because the CSI: Wherever shows had such ludicrous on-screen technobabble, that they couldn’t really do the things those shows have them doing. Nope. It’s a thing, and it works insanely well, and it will be here forever.

This is absolutely not what I want to hear. Does this mean the AI will soon be able to track every single human at all times? So who needs to wipe out the humans? Just eliminate the troublemakers.

Artificial intelligences already track every single human at all times. No matter what laughably weak measures they take to conceal their identities, Steve.

(Oh man I hope your name is Steve. I bet you’d be shitting yourself!)

1 Like

Well, it’s 98% accurate on some data set. Probably a data set with lower correlations than a family album. Also, 98% is fairly low accuracy - in the sense of how a typical person would experience it - even if it is close to the state of the art. (I don’t know the numbers for facial recognition, for MNIST https://en.wikipedia.org/wiki/MNIST_database the best error rate is at or below 0.23%)

1 Like

They forgot step five… “Determine if they are a cop”.

Nooooooooo!

2 Likes

Depending on the quality of the data set 98% is somewhat low based on how you would perceive it organizing your photo album, but it is way way better than humans on average. You can probably recognize a couple hundred people at most with that level of accuracy.

I was going to claim that this is probably not true, but I googled around a bit and found a reported accuracy rate for humans (on still images) a bit above 97%. Still, this does not jibe very well with my day-to-day experience; we would be confusing people with each other on a daily basis. I suppose that my perceived better accuracy may have something to do the fact that my usual interactions with others are 3-dimensional time series with audio rather than 2-dimensional stills.

That’s a lot more data.

Yeah, [quote=“knappa, post:10, topic:82089, full:true”]
I was going to claim that this is probably not true, but I googled around a bit and found a reported accuracy rate for humans (on still images) a bit above 97%. Still, this does not jibe very well with my day-to-day experience; we would be confusing people with each other on a daily basis. I suppose that my perceived better accuracy may have something to do the fact that my usual interactions with others are 3-dimensional time series with audio rather than 2-dimensional stills.
[/quote]

Yes. The photos are not necessarily ideal images – they might catch someone in profile or partially occluded. In real life, you can then try to take a second look, but with a photo you don’t have that option. Photos also loose a lot of context – you tend to see the same people in the same places, and you might not immediately recognize someone you are familiar with in an unfamiliar context.

1 Like

What happens to a face that’s not in the system? Does it result in, “We don’t know who this lady is, but we’ve marked all her photos as being the same person, and we’ll attach a name to it as soon as we figure out who she is.” Or does it just label that person as unknown?

In the parlance of the link, I’m talking about the step “compare to known faces,” when that face is unknown.

Well, we all turn into our parents eventually.

1 Like

What!? A problem can be decomposed into smaller, simpler, more tractable problems? It’s almost as if someone were using math!

The former. Or really, the algorithm (the important part) is not classifying faces against a known database of possibilities. Instead, it is using a neural network to compute a list of numbers (a point in 128 dimensional space in this example) such that pictures of the same face generate coordinates very close together, while pictures of different faces are very far apart. So now we just have to group nearby points and say they are “the same person”, but the classifier itself doesn’t have an idea of a ‘reference image’ for each person that it is trying to match against. A new person will just generate a new classification vector that isn’t close to any existing data points, and the higher level application will decide it is a new person not yet associated with any profile.

Like many neural network algorithms, it can be quite powerful, but is heavily dependent on a good training dataset – particularly one that is very large and very diverse. Just like humans, an algorithm that has mostly only seen white europeans will tend to have more classification error on asians and vice versa.

This topic was automatically closed after 5 days. New replies are no longer allowed.