Face Recognition

A face gets reduced to a bundle of numbers

Recognition starts by cropping out the face and correcting its size and tilt. Points get marked at the corners of the eyes and mouth, then the photo gets rotated and stretched until those points land in roughly the same spot every time. Faces have to be lined up like this before they can be compared at all.

Then comes the key step. The tidied-up face gets squeezed down into a bundle of a few hundred numbers. These aren't values a person could name, like the distance between the eyes — they're values shaped during training so that the same person's faces land close together and different people's land far apart.

That's what lets the bundle hold steady when lighting changes or someone puts on glasses. And matching never has to look at the photo again — just how close two bundles of numbers sit. A roster of tens of thousands of people gets scanned almost instantly.

There are two kinds of question

The first is verification: "is this the person who registered." It pulls exactly one entry off the roster and compares. Unlocking a phone or checking someone through a door falls here. The answer is a plain yes or no, and one comparison keeps it fast and error-resistant.

The second is identification: "which person on the roster is this." It compares against everyone registered and finds the closest match. The number of comparisons grows with however many people are on the roster.

Identification is far harder than verification. As the roster grows, so do the odds that a mere lookalike slips in. Even with identical matching accuracy, picking someone out of ten is a different story from picking them out of a hundred thousand.

How close counts as the same person

Once the distance between two bundles of numbers comes out, judgment takes over. A person decides how close is close enough to call it the same face. Where that line sits shapes almost everything about how the system behaves.

Set the threshold loose and fewer real people get locked out, but more lookalikes get waved through. Set it strict and strangers get blocked reliably, but the real person ends up retrying more often. There's no way to shrink both mistakes at once — moving the line just trades one for the other.

Bias compounds the problem here. If certain skin tones or age groups were underrepresented in the training faces, that group sees more errors at the very same threshold. The same door ends up easy for some people and a constant struggle for others.

Who built the roster, and how

What sets face recognition apart from other matching technologies isn't accuracy — it's that a roster to compare against has to exist first. No matter how good the technology is, it can't find someone whose face was never registered.

So the real questions shift toward the roster: did the person know and agree to being added, how long is it kept, and does it ever get used beyond the stated purpose. A face collected as a badge substitute, later used to trace someone's path through a store, stays the same system while quietly stepping outside what was agreed to.

A face can't be reset the way a password can, and that weighs heavily too. A leaked password gets replaced with a new one; a leaked face follows a person for life. A face can also be captured from a distance without anyone noticing — unlike a fingerprint, a roster can be built without the person ever realizing it.

Many places treat this data as unusually sensitive for exactly this reason, putting special limits on collecting faces in public without consent. Before the technology gets used at all, the question worth asking first is why this roster needs to exist.