What is Pose Estimation? — one analogy

1The analogy

A store mannequin has shoulders, elbows, a waist, and knees built to rotate. When a clerk dresses it in new clothes and sets a pose, all they're doing is turning each joint to a chosen angle. Arm raised or leaning to one side, every pose ultimately comes down to where the joints sit and how sharply they're bent relative to each other.

Pose estimation runs that same process in reverse. Given a single photo where the pose is already set, it works backward to guess where the joints must have been to produce exactly that look, marking each with a point. Instead of turning a mannequin to build a pose, it looks at a finished pose and figures out the joints.

Connect the marked points in a fixed order and what's left is a skeleton made of a few bars. Clothing color, background — all of it disappears, and only the joint locations remain. That skeleton is what pose estimation hands back.

2In detail

The spots to mark get listed first

Pose estimation doesn't mark points just anywhere. The nose, both eyes, both shoulders, both elbows, both wrists, both hips, both knees, both ankles — which spots to mark is fixed on a list ahead of time. A common list runs around seventeen points, and some lists push past thirty by adding finger joints or toes.

A fixed list means a fixed shape of result, too. Whatever photo goes in, the same number of points always comes back, and the same numbered point always means the left elbow. A program reading that result can trust that "point three is the shoulder" and build its logic on top.

What the model actually does is closer to scanning the whole photo and scoring every location for "how likely this is the left shoulder." It picks the highest-scoring spot, marks one point there, and moves to the next item on the list.

Which points connect is already mapped out

Scattered points alone are hard to read as a pose. So which pairs to connect — shoulder to elbow, elbow to wrist — is fixed in a table. Draw lines following that table and a skeleton shape appears.

Once there are lines, angles can be measured. Measure the gap between the line from shoulder to elbow and the line from elbow to wrist and you can tell whether the arm is straight or bent. The same goes for how bent a knee is or how tilted a back is. That angle is exactly what lets a fitness app say something like "your back is bending too far forward."

Body size and camera distance vary from person to person, but angles barely budge. That's why comparing and judging poses relies more often on the angle and length ratio between points than on the raw location of a point.

With several people, points need to be grouped

Two people standing in one scene means four shoulder points and four knee points come out. Unless those points get grouped by whose body they belong to, an arbitrary result appears — one person's shoulder wired to a different person's wrist in the same skeleton.

There are broadly two ways to group them. One finds the box around each person first, then marks points inside each box separately. It's clean with few people, but slows down as the crowd grows. The other marks every point across the whole scene at once, then works out afterward which points belong to the same body. Speed holds up better in crowded scenes.

This grouping wobbles when two people overlap or cross their arms. Miscounting how many people are present, or a few points sticking to the wrong neighbor, both happen at this stage.

Hidden joints get a guessed point too

Even with legs hidden behind a desk, a knee point still comes out in the result. Whatever's on the list tends to get filled in regardless. A spot that can't be seen gets a point placed somewhere plausible, based on the layout of the points that are visible. That's possible because how body proportions work and how far joints can bend got absorbed during training.

The model attaches a confidence value to every point in exchange. A clearly visible shoulder gets a high value; a hidden knee gets a low one. Ignore that number and trust every point equally, and the judgment goes wrong. Real services either skip drawing points below a set level or drop them from the calculation entirely.

Video lets tools smooth points further by looking at neighboring frames. Comparing a point that suddenly jumped in one frame against its neighbors and easing it back in line makes the skeleton shake less.

3More precisely

What pose estimation hands back is a horizontal and vertical coordinate plus a confidence value for every joint. Output limited to the flat screen plane is called two-dimensional pose estimation; add how far each point sits from the camera and it becomes three-dimensional. The three-dimensional kind struggles with front-to-back distance from a single camera, so it either uses multiple cameras or leans on assumptions about body proportions to fill the gap.

Rather than marking a point directly, the more common computation divides the screen into a grid, fills it like a brightness map with "how likely this is the left elbow" at every cell, and picks the brightest spot.

The analogy breaks down somewhere too. A mannequin's joints really do sit in that spot and only rotate in fixed directions, but pose estimation never sees a joint directly — it only sees brightness and pattern on screen and guesses the location from that. So in loose clothing, dim lighting, or a sideways-lying pose — situations that were rare in training — a point can land where there's no joint at all. A mannequin's arm won't bend backward, but an estimated result sometimes comes out at an angle no human body can actually make.

4Try it yourself

5Common misconceptions

It's easy to think pose estimation also figures out who the person is, but actually it only marks joint locations and has nothing to do with identity — the points alone can't even tell if it's the same person.
It's easy to think a point being marked means a joint is really there, but actually hidden spots get filled in by guesswork too, so the confidence value needs checking alongside it.
It's easy to think the meaning of the movement comes along with it, but actually only the point locations come out — a judgment like "waving a hand" is made by a later step that receives those points.

7One-line summary

In shortPose estimation marks joint locations in a photo with points and connects them in a fixed order, turning a pose into a handful of points and lines.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02