Face Detection
Drawing a box around every face in a photo
- Face detection finds whether a face is there, how many, and where in a photo or video, and marks each one with a box.
- The output is a box's position and size, plus a confidence number. It never says who the face belongs to.
- The screen gets checked spot by spot for something face-shaped, so catching both a tiny distant face and a big close one takes several passes at different sizes.
- When several boxes land on the same spot, only the most confident one survives the cleanup.
- It is the starting point for everything that follows — autofocus, smile detection, blurring a face out.
Contents
1The analogy
Nobody cuts fabric straight off the bolt. A dressmaker lays a paper pattern piece over the cloth first, sliding it around until the grain lines up, then traces chalk around the edge before any scissors come out.
There isn't just one pattern piece, either. The same shape comes in several sizes — a big piece for a large panel, a small one for something like a sleeve. Cutting several pieces from one bolt of cloth leaves chalk marks scattered all over it.
Face detection is that chalk mark. It slides across a photo bit by bit, looking for a spot that fits a face-shaped template, and draws a box wherever one fits. It never asks who is inside that box.
2In detail
It scans the screen bit by bit
Face detection doesn't take in a photo all at once. It puts a small window in a corner and asks "does this look like a face," then nudges the window over and asks again, working across the whole image. A narrower step covers more ground but takes longer.
Newer approaches skip that window-by-window crawl. They divide the image into a grid instead, and every cell answers at once: how likely a face sits here, and if so, where the box's edges fall. One pass over the whole image finishes the job — much faster.
That scan alone spits out hundreds of candidate boxes. Most carry such low confidence they get dropped right away; only the ones clearing a threshold move on.
It checks several sizes
The same face fills a tiny patch of screen from far away and a big patch up close. A single fixed window size only ever catches one of those. So detectors either shrink the photo through several sizes and repeat the same scan, or keep several template sizes ready and try them all at once.
This is where speed gets traded off. More size steps catch smaller faces but multiply the computation. A webcam tool that has to run live often keeps the steps few and simply gives up on very small faces.
Step back from a demo tool and at some point the box vanishes — that's the moment your face got smaller than the smallest template on hand.
Overlapping boxes get cleaned up
A single face rarely gets exactly one box. Slightly different positions and sizes all land hits at once — the same face to a human eye, but separate candidates to the math.
So a cleanup pass runs at the end. Keep the box with the highest confidence first, then drop any other box that overlaps it heavily, treating it as the same face. Repeat by picking the next-highest survivor, and one face ends up with one box.
How much overlap counts as "the same face" is adjustable. Set it too strict and two people standing close together merge into one; too loose and a single face keeps two or three boxes.
Whether, not who
All detection hands back is position, size, and a confidence number. Who is inside that box is not detection's job. Naming the person is the next stage, face recognition.
That split matters. Autofocus on a camera app, blurring the background in a video call, blurring a face in footage — most of the features people use daily only need the location. They never generate, and never keep, anything that could identify someone.
Detection is also the first step behind nearly everything else that touches faces. Marking the corners of the eyes and mouth, guessing an expression or an age — all of it starts by cropping the box around a face first.
3More precisely
Face detection is a branch of object detection with one fixed target. Many tools throw in a few extra points — eye, nose, and mouth locations, or how far the face is turned — alongside the box. The box itself is usually given as a top-left corner plus width and height, expressed as a fraction of the image so it still works no matter the photo's size.
Performance is described through two kinds of mistakes: missing a face that's there, and flagging something that isn't a face at all. Lowering the confidence threshold shrinks the first kind of mistake and grows the second. Which one hurts more depends on the use, so the threshold gets set differently service by service.
The analogy breaks down somewhere too. A pattern piece is paper cut to an exact shape — it either fits or it doesn't. A detector isn't holding a fixed template up against the image at all; it scores how face-like a spot is, learned from countless examples during training. That's why wallpaper patterns or an electrical outlet, arranged just so, sometimes get boxed. It's also why a face turned far sideways, covered by a mask, or lit poorly gets missed — those situations were simply rare in training. If certain skin tones or age groups were underrepresented in the training data, misses skew more often toward those groups too.
4Try it yourself
5Common misconceptions
It's easy to think finding a face means knowing who it is, but actually detection only returns a position and size — figuring out identity is the separate job of face recognition.
It's easy to think no box means no face, but actually angle, lighting, and size make real faces slip through fairly often.
It's easy to think only human faces get boxed, but actually a face in a poster, a doll, or an object with a similar pattern can get one too.
7One-line summary
In shortFace detection scans a photo at several sizes and draws a box wherever a face sits, without ever asking who is inside.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02