What is Positional Encoding? — one analogy

1The analogy

A theater ticket has a seat number printed on it. Without that number, what happens? The audience is all seated, but there's no way to tell who's in the front row and who's in the back. Same headcount, same faces, only the order is gone. Positional encoding is what prints that seat number onto the ticket.

The number doesn't travel separately from the audience member. One ticket carries the seat number and the ticket holder's information together. Positional encoding works the same way — it adds an order tag directly into a piece's content, folding both into one block.

The moment a seat number is attached, a relationship appears: "the front row is closer to the stage than the back row." Distance and order only become sayable once numbers exist.

2In detail

A structure that loses track of order

The older method of passing a sentence one word at a time, in order, had no order problem — the processing order was the order. It was just slow.

The method used now has every spot look at every other spot at once. That made things much faster, but it came at a cost: with everyone looking at everyone simultaneously, there's no way to tell what came first. "The dog chased the cat" and "the cat chased the dog" would look like the identical bag of words.

So an order tag has to be attached before the sentence goes in. Without it, scrambling word order however you like wouldn't change the model's output at all.

The number gets added into the content

The tag isn't attached beside a piece — it's off to the side in name only. A number bundle representing order, the same size as the content bundle, is added into it to make one combined value. That spot's value now carries both "what it is" and "which position it is" together.

Adding them together works fine because there's plenty of room to hold both. The content bundle has hundreds to thousands of entries, giving content and order room to settle into different directions within it, and training teaches the model to read the two apart.

Two ways to build the tag

The method first proposed built the tag from a fixed, human-designed rule — layering waves of several different frequencies so each position ends up with its own distinct pattern. It needs no training and can assign numbers even to lengths longer than anything seen during training.

The other approach leaves the tag itself to be learned. A number bundle is placed at each position and its values get tuned during training. This produces a tag well-suited to the data, but it has no number to hand out once you go past however many positions were prepared in advance.

Distance matters more than the absolute number

Writing down the raw position number has a weakness. The same sentence gets a completely different number depending on whether it sits near the start or the end of a document, when what usually matters is really "how many spots apart are this word and that word."

That's why the method widely used now makes the distance between two spots show up directly in the calculation — values get rotated slightly based on position number, so comparing two spots naturally reflects the gap between them. Read the same gap anywhere in the text and it comes out as the same relationship, which makes things far more stable over long text.

Running out of seats

A model gets shaky when it sees a position number far beyond anything encountered during training — like being handed a ticket for a seat the theater doesn't have. Sentences can suddenly turn awkward, or the beginning gets lost.

This is why widening a context window isn't simply a matter of raising a number. The tags need to be rescaled for the wider range, and some further training at the new length is usually needed too.

3More precisely

Positional encoding adds a vector carrying position information into each token's embedding, or folds position relationships into the attention computation itself. The method first proposed used fixed sine and cosine waves of several frequencies laid out differently at each position; learned vectors at each position were also widely used. What most large models rely on now is a rotary approach, which rotates the query and key by an amount tied to position, so the relative distance between two spots comes through naturally.

The analogy breaks down in places. A theater seat number is a single integer, while positional encoding is a bundle of hundreds of values at each spot. A seat number is printed apart from the audience member's information, while positional encoding is added directly into the content value and isn't visibly separable from the outside once the two are combined. And theater seat numbers can't repeat, while position tags only need to be distinct from each other — they don't even need to increase in order, or resemble ordinary counting at all, once training has had its say.

4Try it yourself

Transformer Explainer (how it works) ailearn.space Watch, step by step, exactly where an order tag gets added right before the input reaches the model

5Common misconceptions

It's easy to think a model naturally knows word order, but actually shuffling the words produces the identical result unless an order tag is attached separately.
It's easy to think the position tag rides alongside a piece separately, but actually it's added directly into the content value and travels as one block.
It's easy to think context length grows just by changing a setting, but actually the position tags need to be rescaled for the wider range and some further training is needed too.

7One-line summary

In shortPositional encoding is like a theater seat number tagging each piece with its place in line, handing back a sense of order and distance to a structure that otherwise has none.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02