Key points
- Face detection finds whether a face is there, how many, and where in a photo or video, and marks each one with a box.
- The output is a box's position and size, plus a confidence number. It never says who the face belongs to.
- Every spot on the screen gets scored for how face-like it looks. These days the whole image is computed in one go, with an answer at each spot. Catching both a tiny distant face and a big close one means looking at several sizes.
- When several boxes land on the same spot, only the most confident one survives the cleanup.
- It is the starting point for everything that follows — autofocus, smile detection, blurring a face out.
The analogy
Nobody cuts fabric straight off the bolt. A dressmaker lays a paper pattern piece over the cloth first, sliding it around until the grain lines up, then traces chalk around the edge before any scissors come out.
There isn't just one pattern piece, either. The same shape comes in several sizes — a big piece for a large panel, a small one for something like a sleeve. Cutting several pieces from one bolt of cloth leaves chalk marks scattered all over it.
Face detection is that chalk mark. It slides across a photo bit by bit, looking for a spot that fits a face-shaped template, and draws a box wherever one fits. It never asks who is inside that box.
In detail
Every spot gets scored for how face-like it is
The older approach didn't take in a photo all at once. It put a small window in a corner and asked "does this look like a face," then nudged the window over and asked again, working across the whole image. A narrower step covered more ground but took longer.
Newer approaches skip that window-by-window crawl. They divide the image into a grid instead, and every cell answers at once: how likely a face sits here, and if so, where the box's edges fall. One pass over the whole image finishes the job — much faster.
That scan alone spits out hundreds of candidate boxes. Most carry such low confidence they get dropped right away; only the ones clearing a threshold move on.
It checks several sizes
The same face fills a tiny patch of screen from far away and a big patch up close. A single fixed window size only ever catches one of those. So detectors either shrink the photo through several sizes and repeat the same scan, or keep several reference box sizes in place and score them all at once.
This is where speed gets traded off. More size steps catch smaller faces but multiply the computation. A webcam tool that has to run live often keeps the steps few and simply gives up on very small faces.
Step back from a demo tool and at some point the box vanishes — that's the moment your face came across so small that its face-likeness score dropped below the cutoff. It happens especially once the face gets smaller than the smallest size step.
Overlapping boxes get cleaned up
A single face rarely gets exactly one box. Slightly different positions and sizes all land hits at once — the same face to a human eye, but separate candidates to the math.
So a cleanup pass runs at the end. Keep the box with the highest confidence first, then drop any other box that overlaps it heavily, treating it as the same face. Repeat by picking the next-highest survivor, and one face ends up with one box.
How much overlap counts as "the same face" is adjustable. Set it too strict and two people standing close together merge into one; too loose and a single face keeps two or three boxes.
Whether, not who
What detection hands back is mainly position, size, and a confidence number — some tools tack on a few spots for the eyes, nose, and mouth. Who is inside that box is not detection's job. Naming the person is the next stage, face recognition.
That split matters. Autofocus on a camera app, blurring the background in a video call, blurring a face in footage — most of the features people use daily only need the location. The detection step itself never works out who anyone is. A photo with a face in it is identifying information on its own, though, and whether a service stores or sends it varies from one service to the next.
Detection is also the first step behind nearly everything else that touches faces. Marking the corners of the eyes and mouth, guessing an expression or an age — all of it starts by cropping the box around a face first.
More precisely
Face detection is a branch of object detection with one fixed target. Many tools throw in a few extra points — eye, nose, and mouth locations, or how far the face is turned — alongside the box. The box itself is given as a top-left corner plus width and height, or as two opposite corners. Some tools express it as a fraction of the image so it still works no matter the photo's size.
Performance is described through two kinds of mistakes: missing a face that's there, and flagging something that isn't a face at all. Lowering the confidence threshold shrinks the first kind of mistake and grows the second. Which one hurts more depends on the use, so the threshold gets set differently service by service.
The analogy breaks down somewhere too. A pattern piece is paper cut to an exact shape — it either fits or it doesn't. A detector isn't holding a fixed template up against the image at all; it scores how face-like a spot is, learned from countless examples during training. That's why wallpaper patterns or an electrical outlet, arranged just so, sometimes get boxed. It's also why a face turned far sideways, covered by a mask, or lit poorly gets missed — those situations were simply rare in training. If certain skin tones or age groups were underrepresented in the training data, misses skew more often toward those groups too.
Try it yourself
Common misconceptions
It's easy to think finding a face means knowing who it is, but actually detection only returns a position and size — figuring out identity is the separate job of face recognition.
It's easy to think no box means no face, but actually angle, lighting, and size make real faces slip through fairly often.
It's easy to think only human faces get boxed, but actually a face in a poster, a doll, or an object with a similar pattern can get one too.
One-line summary
Face detection weighs every spot in a photo, at several sizes, for how face-like it looks and draws a box wherever a face sits, without ever asking who is inside.
Spotted an error or have a better analogy? Suggest an edit · Last updated 2026-10-17