What is Labeling? — one analogy

1The analogy

A museum storeroom holds rows of boxed artifacts. Without a tag noting where each one came from and what it was used for, even a priceless piece can't be found or compared later. So someone opens each box and attaches a tag by hand.

The work looks simple until it hits a boundary. A piece that could be a bowl or a cup, a shard too broken to tell what it once was — these keep coming up. Two people looking at the same box sometimes reach for different tags.

So a guide hangs in the storeroom, spelling out the tagging rules: this shape gets written this way, anything ambiguous goes in a holding pile. Where judgments still split, two people tag independently and only the mismatches get discussed again.

Labeling is exactly this work. Because the tags attached become the standard for everything after, how they're attached matters far more than how fast.

2In detail

Attaching answers is really the work of building a standard

It's easy to treat labeling as mere prep work, but it's actually where what counts as correct gets decided. Whichever way something got labeled becomes exactly what the model learns, and later, the yardstick it gets graded against.

So a few things need deciding before labeling ever starts. What the categories are, what happens to something that doesn't fit any of them, and whether one item can carry more than one answer. Skip these and dive in, and there's a good chance of starting over halfway through.

The guide is half the job

A well-built guide isn't just a list of category names. It carries worked examples right on the boundary — this case goes here, that case goes there, shown with real data.

A guide is never finished on day one, either. It grows as tricky cases come up during the work, each judgment call getting written back into it. Early on, a small batch gets labeled to shake the guide out, and only once the rules settle does the bulk of the work begin.

Change the guide, and earlier work needs revisiting to match. A batch of data where the early part and the later part follow different standards is hard to train on and hard to grade.

Different people label differently

Hand the same data to two people, and some share of disagreement is guaranteed. Measuring how much they disagree tells you something about the state of the data itself. Low agreement doesn't mean the workers were careless — it usually means the guide is vague, or the data itself is ambiguous.

That number sets expectations too. If people themselves only land on the same answer five times out of seven, asking a model to beat that is asking for more than the data can support.

Don't dump the ambiguous cases — collect them

Shoving a borderline item into whichever bin is convenient feels efficient in the moment, but it costs later. The model ends up making its most frequent mistakes right in that ambiguous zone, and because the answer itself was shaky, there's no clear direction to fix it in.

So ambiguous cases get pulled aside into a holding pile and discussed together. The calls made there become the most valuable part of the guide. And that leftover pile comes in handy again later, as a test set for checking exactly where a model tends to get confused.

Automating the first pass still leaves a review step

Once data grows past what human hands can handle, other approaches get mixed in — rules that attach labels automatically, or an already-trained model drafting a first pass that a person then corrects. Sending only the low-confidence cases to a person is common too.

Whatever the approach, a review step by a person stays in the loop. Automated labeling tends to make the same mistake in the same direction over and over, and that kind of consistent error does far more damage than a random one would. Pulling a sample of the labeled results for a person to re-check belongs in the schedule from the start, not bolted on later.

3More precisely

Labeling comes in several shapes. Attaching one category to an entire piece of data, boxing in where an object sits inside an image, painting a region pixel by pixel, linking how words in a sentence relate to each other — the shape of the work changes with the goal. The more hands-on a shape is, the more each item costs.

The guide isn't the only thing that affects quality. A reviewer's standard tends to drift the longer a single session runs, and reviewing a run of similar items back-to-back creates its own lean, where earlier judgments pull on later ones. That's why the order gets shuffled, and known answers get slipped in partway through as a check.

The analogy breaks down in one place. A museum artifact is a physical object — if a tag looks doubtful, it can be pulled back out and checked. A label attached to data can't be revisited the same way unless the reasoning behind it was written down at the time. It's worth keeping a record of why a close call went the way it did.

4Try it yourself

Teachable Machine ailearn.space Sorting photos into bins is labeling itself. Try deciding for yourself which bin an ambiguous photo belongs in.

5Common misconceptions

It's easy to think labeling is simple grunt work, but actually it's where what counts as correct gets decided, which sets the ceiling on how well a model can ever do.
It's easy to think more labeled data is always better, but actually data piled up under a shaky standard ends up confusing both training and grading.
It's easy to think automated labeling removes the need for people, but actually automated work leaves consistent errors in the same direction, which makes review more important, not less.

7One-line summary

In shortLabeling is the work of a person attaching an answer to each piece of data, and how the boundary cases get decided matters more than how fast the work goes.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02