Multilayer Perceptron

What one unit does

Once you see what a single station does, everything else follows easily. A unit takes in a batch of numbers from whatever came before it. It multiplies each number by its own set weight — a dial that decides how much that particular input matters. Some inputs get a large weight, others get one close to zero.

Adding up all those products gives a single score. To that score, the unit adds one more number it keeps for itself: a baseline. That baseline sets the unit's personality — "stay quiet unless something strong shows up" or "react to even the faintest signal."

Without a bend, stacking layers is pointless

Passing that score straight up to the next row causes a problem. Repeating multiply-and-add over and over still ends up mathematically identical to doing it once. Stack a hundred rows and you'd get nothing more than a single row could do.

So at the end, the score gets bent, once. The most common trick is to flatten every negative value to zero and let positive values pass through unchanged. That single kink is what makes stacking rows meaningful at all — layer enough kinked lines together and you get something close to a curve.

Later rows judge only on earlier scores

The first row of stations looks straight at the object. From the second row on, no station sees the object at all — it only sees the scores the row before it wrote down. A row that works only in the middle, invisible from the outside, is called a hidden layer.

Every unit connects to every single unit in the row before it, with nothing skipped. Ten stations in the row before means the next row's unit receives all ten scores. Which of them matters more is decided entirely by the multiplied weight. That's why this kind of network is called fully connected.

The signal only ever flows forward. There's no path for a later row to ask an earlier one a question again. From the moment the object comes in to the moment a result comes out, everything moves straight through in one direction.

Correcting by how far off the answer was

Nobody sets the weights by hand. They start out scattered at meaningless values. Feed in a photo of a can and get "plastic bottle" back, and the network measures how far off that was as a number, then nudges the weights a little, working backward from the end to the start.

Repeat that correction hundreds of thousands of times, and each station settles into checking whatever it turns out to be useful for. Nobody has to write down "check for shine" or "check for see-through" in advance — the network finds, on its own, whatever cues are useful for telling things apart.

Still built into today's large models

A multilayer perceptron isn't some outdated structure. Open up a network that handles images, or a large language model that handles text, and this structure is tucked into every layer. When something like attention gathers information together, the part that actually does something with it is still made of units that multiply, add, and bend once.

A large share of a big model's total weight often sits right here. The structure is simple, so the math runs fast, and its capacity is easy to scale up — just add more units per row.