Parameter
A tuning dial inside a model whose value training decides
- A parameter is a number inside a model whose value training decides on its own. Nobody types these in by hand, one at a time.
- There are broadly two kinds of dial: one that multiplies an incoming value, and one that shifts the whole result up or down.
- Training is the process of turning the dials by a tiny amount, over and over, in proportion to how wrong the answer was.
- The number of parameters is the model's size. Storage space and required memory both trace back to this count.
- A value a person sets ahead of time isn't a parameter — that's a hyperparameter. The two get mixed up often.
Contents
1The analogy
The flavor of a food stall's signature sauce doesn't come down to the ingredients — it comes down to the ratios. Picture a row of dials on the counter, each controlling how many spoonfuls of chili powder, how many spoonfuls of sugar, how many cups of water go in. At first they're all set to arbitrary positions. Someone tastes a spoonful, and if the verdict is "too spicy," the chili-powder dial gets nudged slightly to the left; if it's "too bland," the sugar dial gets nudged slightly to the right. Taste, adjust a little, taste again — repeat that a few hundred times, and the result is that stall's recipe. Parameters are these dials. The only difference is there are hundreds of millions, even hundreds of billions of them, so training turns them instead of a person.
2In detail
Training is what turns the dials
When a model is first built, its dials are scattered at meaningless values. Ask it to solve a problem in that state and, naturally, the answer comes out wrong. Training measures how wrong, works out which dial should turn which way and by how much to be less wrong next time, and nudges it just that much.
Not turning any dial too far in one go matters. Turn one hard, and this problem gets solved while an earlier one gets broken again. Turn everything a tiny bit at a time, over and over, and the dials drift slowly toward a spot that handles a wide range of problems reasonably well.
Multiplying dials and adding dials
Dials come in two kinds. One is a value that gets multiplied by an incoming value. A large value here means that input is treated as important; a small one means it's mostly ignored; a negative one means it's read as evidence for the opposite. This dial decides how closely to listen to a given signal.
The other kind gets added once the calculation is done. Multiplying alone can't shift a result up or down as a whole — this dial raises or lowers the baseline itself. It's effectively a dial for how generous or how strict the overall judgment should be.
Multiplying dials vastly outnumber adding dials. They connect every unit in one layer to every unit in the next, so their count multiplies as layers grow wider. Adding dials attach only one per unit in the next layer, so they make up a tiny sliver of the total.
The count is the model's size
When people call a model big or small, what they're measuring is this dial count. More dials mean finer rules can be stored, and room for subtler distinctions. In exchange, more storage space is needed, and more memory has to be set aside to run it.
More isn't automatically better. Too many dials relative to the amount of training data, and the model starts memorizing the data itself instead of learning a general knack for the problem. So the amount of data, the compute available, and the dial count all get balanced against each other. Storing dial values more coarsely, or trimming away dials that barely get used, are common ways to make a model light enough to run on modest hardware.
Some values are set by a person instead
Only values that get set on their own during training count as parameters. Things like how far to turn a dial in one step, how many times to go through the data, or how many layers to stack — a person decides these before training even starts. These are called hyperparameters instead.
The difference is who sets them. A cook decides the heat level and the cooking time ahead of time; the sauce ratios get corrected by tasting along the way. When training isn't going well, the place to look is usually the values a person set in advance.
3More precisely
Parameters are the umbrella term for every number inside a model whose value gets updated during training. Most of them are weights connecting one layer to the next, with a small remainder being biases. Saving a model means writing these numbers to a file, alongside a description of its architecture.
The analogy breaks down in a place too. Counter dials can be turned one at a time, but training shifts every single dial a little bit, all at once. That makes it hard for a person to point at any one dial and say what it's responsible for. A single dial means nothing on its own — meaning only emerges from the combination hundreds of millions of them form together. Dial count also can't be tied directly to performance. The same count can produce very different results depending on the architecture and the training data. And where a counter always produces the same flavor once the dials are set, a model can produce a completely different answer depending on the order and path a calculation takes, even with the same dial values.
4Try it yourself
5Common misconceptions
It's easy to think a person sets each parameter one by one, but actually training shifts the values on its own while looking at the data.
It's easy to think more parameters always means smarter, but actually without enough data and the right architecture to back them up, a model just memorizes and struggles on anything new.
It's easy to think the training data itself is stored inside the parameters, but actually what's left behind isn't the data — it's only the adjustments extracted from it.
7One-line summary
In shortParameters are the dials inside a model that training gradually turns while checking its answers, and their count decides the model's size.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02