What is Autoregressive? — one analogy

1The analogy

There's a chain-story game people play in a group chat. One person adds a line at a time, and the story grows line by line. Except in this particular chat, there's only one person in it.

So they do it alone. They scroll up and reread everything posted so far, start to finish, including the line they just added. Then they decide exactly one more line to post. The moment it's posted, they scroll back up and read from the start again, and post one more line.

That loop is autoregressive. A line posted in the chat can't be deleted. If the story wanders off somewhere strange on line three, every line after that treats the strange turn as an established fact and keeps building on it.

2In detail

It settles exactly one piece at a time

What a model actually does when producing an answer is simpler than it sounds. It reads everything written so far, scores every candidate for how plausible it would be as the next piece, and picks one to attach at the end. That's one step.

Once a step finishes, the text is one piece longer. Now it rereads that longer text from the beginning and does the exact same thing again. If the answer ends up a hundred pieces long, it takes that step a hundred times.

It isn't planning out the whole sentence in advance and dictating it. Even where the sentence ends only gets decided step by step, the moment a period or an end-marker piece happens to get picked.

What it just wrote becomes the next input

This is where the name autoregressive comes from. The model's own output immediately becomes its next input. Nobody feeds it anything new — it just receives back what it made.

That's why the first few pieces shape the whole direction of an answer so heavily. Open with "Sure, here's how" and an explanatory tone tends to follow. Open with "I'm sorry, but" and a refusal tends to follow. This is also why the trick of pre-writing the start of a response inside a prompt works so well.

The same property creates problems too. Once a false name slips into an early sentence, the sentences after it treat that name as an already-settled fact and keep building the story around it naturally. Wrong details get more and more convincing as they go.

It can't be undone

People revise earlier sentences while writing. The autoregressive approach has no such step. Once a piece is attached, it stays, and the model can only think about what comes next on top of it.

This is why some systems keep several candidate continuations running at once and pick the best one at the end. Others let the model finish an answer, then review and rewrite it. A reasoning model writing a long chain of thought before its actual answer is really the same idea: if you can't undo anything, you might as well wander through the thinking first.

Why writing is slower than reading

Reading a question and writing an answer run at very different speeds. A question already exists in full, so every piece of it can be loaded in and processed at the same time. An answer is different — the next piece can't start until the one before it is settled.

That means the whole model has to be run once for every single piece. Dozens of layers, billions of weights, all just to get one piece out. Double the length of an answer and you double the wait. Streaming, where text seems to trickle across the screen, isn't hiding this process — it's just showing every step exactly as it happens.

Other approaches exist

Not every generation method works this way. Diffusion, widely used for images, refines an entire picture a little at a time all at once. The whole canvas improves together from start to finish, with no fixed order. Similar all-at-once approaches are being researched for text too.

Even so, autoregressive generation has held its ground in text for a long time. Training is simpler, and the text it produces holds together from start to finish.

3More precisely

An autoregressive model takes the token sequence produced so far as its condition and outputs a probability distribution over the next token, then draws one from that distribution and repeats. Settings like temperature or top-p are dials for how that draw is made. During training, the correct sentence already exists, so every position can be trained at once — the only extra piece needed is a mask that stops the model from peeking ahead.

The analogy breaks down in one place. A person rereading a group chat scrolls through it with their eyes every single time, but the model caches the results it already computed for the earlier part and only computes the newly attached piece. So it doesn't actually slow down as much as rereading the whole text every step would suggest. Also, a person mentally sketches out the next sentence in advance, but an autoregressive model has no explicit plan beyond the very next piece. There's active work trying to cut down the number of steps too — for instance, having a small model draft several pieces ahead and letting a larger model check them all at once.

4Try it yourself

5Common misconceptions

It's easy to think the AI writes the whole answer in advance and just reveals it slowly, but actually each piece is generated on the spot and sent out one at a time.
It's easy to assume it corrects earlier mistakes on its own once something goes wrong, but actually it can't undo a piece it already wrote — it can only keep building on top of it.
It's easy to think a faster computer would make the answer appear all at once, but actually the front still has to be settled before the back can be, so the number of steps never shrinks.

7One-line summary

In shortAutoregressive generation is a loop of rereading what it has written so far to decide just the next single piece, and it's the source of both an AI's slow pace and its confident-sounding mistakes.

Did this analogy help?

Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02