Key points
- A bounding box is the smallest box that just wraps an object. Whatever box shows on screen is the result.
- One box is written with four numbers — a top-left position (two numbers) plus a width and a height.
- How well it fits is measured by how much it overlaps the correct box — overlapping area divided by combined area.
- The correct box is drawn or checked by a person. If the standard used to draw it wavers, the model wavers right along with it.
- Boxes are usually axis-aligned, so a tilted or curvy object drags along background inside it.
The analogy
You're cutting a new piece of window screen mesh to fit a frame. Cut it bigger than the frame and the edges curl and buckle; cut it smaller and a gap opens up for insects to get in. The goal is a size that fits the frame exactly.
Once it's cut, you need to check the fit — and instead of eyeballing it and saying "close enough," there's a way to put a number on it. Take the area where the frame and the cut mesh overlap, and divide it by the area the two of them cover together. A perfect fit gives you 1, and the number drops fast the further off you are. A mesh that's half misaligned struggles to clear even 0.5, however generously you look at it.
The frame measurement everyone's checking against is something someone measured with a tape and wrote down. Whether they measured the outer edge of the frame or just the inner groove changes the passing mark entirely. A bounding box works exactly the same way.
In detail
One box is four numbers
Writing down a box is simple. Note the top-left corner's position on screen with two numbers, then how far right and how far down it extends with two more, and that's it. Versions that give the top-left and bottom-right corners, or the center point plus width and height, carry the same meaning, but data sets differ in which they use, and mixing them throws the math off.
Writing the numbers as a fraction of the screen size is common — 30 percent in from the left, width equal to 20 percent of the screen, that kind of thing. Written that way, the box keeps tracking correctly even if the photo gets scaled up or down.
The boxes usually used sit flush with the screen's horizontal and vertical axes. So wrapping a box around an object lying at an angle drags a lot of empty background in with it. That's exactly why the next step up is segmentation. Where tilted objects are common, such as aerial photos, a separate tilted box that also records an angle is sometimes used.
Measuring fit by overlap
Checking how well a box fits means comparing it against a correct box that a person drew. Divide the overlapping area of the two boxes by the total area they cover between them, and you get a number between 0 and 1. A perfect overlap gives 1; barely touching gives something close to 0.
How high that number has to climb to count as "correct" is also something a person decides. A threshold of 0.5 is common, and 0.5 is looser than it sounds — a small box covering just the inner half of the correct box scores exactly 0.5 and squeaks through. Where precision really matters, a stricter threshold like 0.75 gets used, or scores get measured at several thresholds and averaged.
In demos, you'll see the box drawn slightly bigger or smaller than the object — that kind of small mismatch usually clears the threshold with room to spare. It looks off to the eye, but the score often doesn't mind at all.
The correct box is drawn by a person
The correct box a model gets compared against is one a person dragged out with a mouse, photo by photo, or one a person checked and fixed after a model drew it first. Ambiguous moments keep coming up while doing this. Does a mug's handle count as part of the box? Should a partly cropped object be drawn as if the off-screen part were imagined in? Does a reflection in a window get boxed too?
If that standard shifts from person to person, what the model learns as "correct" shifts right along with it. That's why data teams pin down detailed rules up front and have several people draw the same photo to check how closely they agree. Tracing back why a model draws boxes in odd places often turns up a wavering standard in the training data.
What a box can't hold
From image classification, which puts one label on a whole photo, up to object detection, which boxes each object, up again to segmentation, which cuts out shapes pixel by pixel — the bounding box is the middle rung of that ladder.
Being in the middle brings both an upside and a downside. It's widely used because it gives position and size without making data too heavy to prepare. But a curvy object, objects overlapping each other, or an object lying at an angle all drag a lot of the wrong thing into the box. Wherever you need to erase a background cleanly or measure area precisely, a box falls short and you have to move up a rung.
More precisely
A bounding box is the smallest rectangle, usually axis-aligned, that wraps an object, written as four values: depending on the data set, a top-left coordinate plus width and height, or the top-left and bottom-right corners. The value that measures overlap between a predicted box and the correct one is intersection area divided by union area; a threshold on that value sorts hits from misses, and scores are averaged per label into one number.
Overlap alone misses some differences, too. Two boxes that don't touch at all score exactly 0 regardless of how far apart they are, so it can't tell a near miss from a wildly wrong guess. Training sometimes uses a separate calculation that patches over this gap.
The window screen analogy breaks down in one place. A screen only ever gets one piece of mesh per frame, but a model puts out hundreds of candidate boxes per photo and commonly filters out the overlapping ones on the same object, keeping just one. And a window frame's measurement is fixed once and for all, but the correct box for an object in a photo is a wavering standard that shifts a little depending on who drew it. Because the rule for what to wrap differs from one data set to another, scores from models trained on different data sets can be hard to compare directly.
Try it yourself
Common misconceptions
It's easy to think everything inside the box is the object, but actually the box is usually axis-aligned, so a fair amount of background gets mixed in with it.
It's easy to assume an overlap score of 0.5 means it's only half right, but actually it's a ratio over the combined area, so for two equal-sized boxes that are merely offset from each other, each has to overlap about two-thirds of itself to reach 0.5. Even so, that bar is loose for precise work, so higher thresholds are used alongside it.
It's easy to think the correct box is a fixed, settled answer, but actually a person set the rules and drew it or checked and fixed it, so the standard varies from one data set to the next.
One-line summary
A bounding box writes an object's wrapping box as four numbers, scores it by how much it overlaps the correct box, and that correct box is one a person set rules for and drew or checked.
Spotted an error or have a better analogy? Suggest an edit · Last updated 2026-10-17