Transformer

No queue

The older approach lined words up and read them one at a time, each passing what it had gathered to the next. Long sentences faded at the front: after twenty handoffs, almost nothing of the opening survived. And since each step waited for the one before it, the whole thing crawled.

The round table fixes both problems. The last word sees the first directly, so distance costs nothing. In "I left the umbrella I bought yesterday on the train," the phrase about leaving reaches straight back and grabs the umbrella. Because every word scores every other simultaneously, the arithmetic splits across thousands of graphics chips. Training on enormous piles of text became practical because of this seating plan.

Matching questions against name tags

Each word at the table carries three things: a question in hand, a name tag on its chest, and a folder in front of it. The question says what it is looking for. The tag advertises what it can offer. The folder holds what it would actually hand over.

The second bank is holding a question about the verb that would explain it. It carries that question around the table and compares it with every tag. The tag on fished matches well and scores high. The tag on closed matches poorly and scores low. Scoring is not an election with one winner. It is a split: seventy to one word, twenty to another, the rest scattered.

Once the split is fixed, the folders are collected in those proportions. A generous helping from the high scorer, a spoonful from the low one, all blended into a fresh sense of itself. The question is the query, the tag is the key, the folder is the value.

Every word does this at the same instant. Four words, four simultaneous rounds of comparison, and a thousand words work exactly the same way.

More than one meeting

Several meetings run in parallel with the same words in the same seats. Each version cares about something different. One tracks which word pairs with which. Another follows what modifies what. A third keeps chasing a name mentioned two sentences back.

A single meeting sees too little. Watch grammar and you miss meaning; watch meaning and you miss grammar. Run eight or twelve versions, let each look somewhere else, then pool the results, and one word holds several relationships at once. If one version wanders off, the others cover for it. Each parallel meeting is called a head.

Without seat numbers, everything scrambles

Facing each other simultaneously also means losing track of order. A round table has no front and no back. "Dog bites man" and "man bites dog" use the same bag of words and mean opposite things. Strip order away and the two look identical.

So before the words take their seats, each is handed a seat number. The number is not a sticker on the outside. It is blended lightly into the word's own sense, which is why the same word registers a little differently depending on where it sits. This step is called positional encoding.

Stacking layers, ending in one word

One meeting settles very little. The words carry their revised senses upstairs and hold the identical meeting again, through dozens of floors. Downstairs the focus is close at hand, links to neighboring words. Upstairs it widens to the argument of a paragraph and the tone of a document. Nobody assigns those jobs. The division emerges during training.

What comes out of the top floor is surprisingly plain: a list of candidate next words, each with a score. After "so I sat on the bank and," the word "fished" scores high and "sprinted" scores low. One candidate is chosen, pinned to the end of the sentence, and the now-longer sentence goes back to the table from scratch. A few hundred rounds of that produce one long answer.

This is also why a name mentioned early in a chat can still be used much later. Every time a new word is chosen, the entire text so far returns to the table. Nothing is memorized along the way; it is all reread each time.