Key points
- Encoder-decoder is a structure that keeps a reading half and a writing half separate. The front half turns the input into a bundle of numbers carrying meaning and context; the back half looks at that and produces a new sentence.
- The encoder looks at the whole sentence in full, front and back together. The decoder writes one piece at a time, moving forward.
- What passes between the two halves isn't the original text — it's a bundle of numbers carrying meaning and context. The literal words never cross over.
- It suits jobs where the input and the output don't share the same shape, like translation or summarizing.
- Some models today use only the encoder half, others only the decoder half — whichever fits the job at hand.
The analogy
Nobody moves house by shoving an entire room straight into the truck. Clothes from the closet, books from the shelf — everything goes into boxes first, and each box gets labeled with what's inside. At that moment, items stop sitting in drawers and become things packed in boxes. Even so, which room and which spot each one came from gets written on the side of the box.
At the new place, the boxes get opened one at a time and everything gets arranged to fit the new rooms. The room sizes and window placements are different, so the old layout can't just get copied over. You look at what's in each box and put it wherever makes sense in this new house.
The half doing the packing is the encoder; the half unpacking at the new place is the decoder. And the boxes crossing between them hold the numbers that carry the meaning.
In detail
The packing side reads the whole sentence at once
The encoder already has the whole input in hand, so it can refer to the sentence in both directions. A transformer encoder literally looks at it all at once. It can process the third word while referring to the tenth word with no trouble at all, because the entire text is already sitting right there.
That freedom matters. Whether "bank" means a riverbank or a financial bank often only becomes clear from something later in the sentence. Since the encoder can see ahead, it can settle ambiguity like this in advance.
After the sweep, each piece of the sentence turns into its own bundle of numbers. That bundle carries what role that piece played within the sentence, too — the same way packing a box means not just tossing items in, but also writing down what's inside on the side.
The writing side builds one piece at a time
The decoder has it differently. It has no way to peek at a part it hasn't written yet. So it looks at only two things — the pieces it has written so far, and the boxes the encoder handed over — and picks the next single piece from those. It appends the chosen piece, rereads from the top, and picks the next one after that.
That's why the decoder carries a blindfold blocking anything ahead of it. During training, feeding it the full correct sentence all at once would let it peek at the next piece and cheat, so anything not yet written gets deliberately hidden.
It even signals when it's done on its own. Once it judges there's nothing left to add, it produces the piece that means "stop here" and finishes.
The trick that let it keep glancing back at the boxes
The earliest version of this structure had a frustrating limit — however long the sentence, everything had to get squeezed into one single box. Moving a house full of belongings with just one box meant whatever went in first got crushed under everything else. That's exactly why longer sentences tended to lose track of what came at the start.
The fix was not merging everything into a single box. Keep every one of the per-piece bundles the encoder produced, and let the decoder, every time it writes one piece, pick out and glance at whichever one it needs right then. That mechanism is called attention.
Translating "Today I read the book my friend gave me yesterday," writing the word for "yesterday" pulls up the box holding "yesterday"; writing the object pulls up the box holding "book." Looking more closely into the boxes that are actually needed means a long sentence gets far less blurry toward the front.
Today, sometimes only one half gets used
Which half is needed depends on the job. Grasping what a sentence means doesn't need the writing half at all — a model with only the encoder is plenty for jobs like classification or search, where the answer comes out short and clean.
The opposite holds for conversation or writing, where text keeps getting continued — a model with only the decoder is what gets used there. It can treat a question it received as though it were already-written text and just keep going from there. Most conversational models today are built this way.
Structures with both halves still fit well wherever the language or length going in and coming out differ substantially, as in translation or summarizing. These days, though, large decoder-only models often handle such jobs too.
More precisely
The encoder turns input pieces into context-aware bundles of numbers; the decoder takes those bundles together with everything written so far and produces a probability for the next piece. Inside the decoder, the channel that looks at its own output and the channel that looks toward the encoder are kept separate, and the process of hiding what's ahead is called masking.
The analogy breaks down in a place worth naming. Items packed into a moving box stay exactly as they were, but the bundle of numbers the encoder produces isn't a stored copy of the original wording, and there's no built-in feature that hands the original text back. And boxes get loaded once and the truck departs, but in an attention-based design the decoder glances back at the encoder's side fresh, every single time it produces one more piece — unlike the earliest designs, this isn't a handoff that happens just once.
Encoder-decoder isn't the name of a specific model — it's a way of arranging parts. The same arrangement can be built with a recurrent network or with a transformer.
Try it yourself
- Transformer Explainer (how a transformer works) ailearn.space Follow a decoder-only model as it looks only at the pieces before it and picks the next piece one at a time
- Fill in the Blank (what a language model actually learned) ailearn.space A model that uses only an encoder (BERT) fills the blank by looking at the words after it as well. Change the words that follow and see whether the candidates change, and you will get a feel for what "looking at both sides" means
Common misconceptions
It's easy to think the encoder stores the sentence like a zip file, but actually it swaps the words for a bundle of numbers that mixes meaning and context. There's no unzipping feature that pulls the original back out exactly, though some research has shown that enough information remains to recover a good part of it.
It's easy to think the decoder only receives the encoder's output once, but actually in today's attention-based designs it looks back at the encoder's side fresh every single time it produces one more piece.
It's easy to think every language model has both an encoder and a decoder, but actually a large share of today's conversational models are built with only a decoder.
One-line summary
Encoder-decoder splits the job into a half that turns the input into meaning and context, and a half that looks at that meaning and writes a new sentence one piece at a time.
Spotted an error or have a better analogy? Suggest an edit · Last updated 2026-10-18