Key points
- Training is the process of watching examples and nudging a set of dials a little at a time, by exactly as much as the answer was off. It's learning by doing, not by being told.
- It never corrects by a large amount at once. Very small changes, made a very large number of times is the whole trick.
- Once training stops, the dials freeze wherever they landed, and that frozen state, saved, is the model.
- Training is the engine that makes machine learning run, and machine learning is, in turn, one way of building AI among others.
- Repeating the same examples for too long doesn't keep making things better. Past a certain point, it starts just memorizing what it's seen.
The analogy
Picture the day you learned to ride a bike. No matter how carefully someone explains it from the side, it doesn't help. Memorize exactly how many degrees to turn the handlebars and how far to lean your body, and the moment you sit on the seat, you fall over anyway.
Learning happens a different way. Push the pedal, lean right, plant a foot, stop. Climb back on and lean a little less this time. Fall again, correct a little more. Repeat that a few dozen times, and at some point your body starts balancing on its own. You still can't put into words exactly what you corrected or how.
What matters here is the size of the correction. Lean too far the other way after falling to the right, and you fall to the left instead. Correcting back only by exactly as much as you were off, and repeating that tiny correction relentlessly — that's exactly how training happens too.
In detail
One loop is four steps
Training repeats the same four steps over and over. First, feed it an example (usually a small bundle of them) and get an answer out of it. Second, measure how far that answer is from the correct one as a single number. Third, work out which dial should turn which way to bring that number down. Fourth, turn it that way by a tiny amount.
Then it moves to the next example and runs the same four steps again. On a bike, that's the same as riding, feeling how much you leaned, knowing which way to correct, and correcting a little.
This one loop runs over and over, countless times, and for a large model hundreds of thousands of times or more. What changes in a single loop is too small to even notice, but that tiny change piles up until what once said anything at all starts producing a reasonably plausible answer.
Why the correction stays small
There's a real reason you never correct by a large amount at once: the one example you just looked at doesn't represent everything. Swing hard in a new direction because of one strange example, and everything built up so far gets thrown out of balance all at once — the same as jerking the handlebars hard after leaning right and falling the other way instead.
So the size of each step gets set very small on purpose. This step size is called the learning rate. Too large, and it wobbles back and forth around the right answer without ever settling; too small, and getting there takes forever. Finding the right size is one of the important jobs for whoever is running the training.
In practice, models rarely look at examples one at a time; looking at them bundled together, from dozens to thousands at once, is the norm. Averaging over a bundle keeps a single strange example from throwing the direction off, which steadies the whole walk.
The size of the gap gets measured as a single number
For training to run at all, it needs to know, as a number, how wrong things currently are. The ruler used to measure that gap is called the loss function. Match the correct answer exactly and it sits near zero; drift further away and it grows.
Training has exactly one goal: shrink that number. There's no instruction anywhere that says "draw well" or "speak well." A person's job stops at deciding what to use as the ruler — everything after that flows on its own toward shrinking that number.
Which is exactly why picking the wrong ruler sends things somewhere strange. On a bike, measuring only "did you avoid falling" teaches it to just stand still. This exact kind of accident happens surprisingly often in real training.
Training longer doesn't keep helping forever
Keep training running, and the loss keeps shrinking. But if the same examples get repeated too many times, something strange can start happening past a certain point. It keeps getting better and better at the examples it trained on, while possibly getting worse on examples it's never seen. Instead of finding the rule, it started memorizing what it saw. This state is called overfitting.
So a separate set of examples gets held back from training and checked periodically. A common approach is to stop training right there, once that score stops improving and turns the other way. It's similar to how looping the same street after you've already stopped falling doesn't build skill — you have to actually go somewhere new for that.
Once training ends, a model is left behind
The moment training stops, the dials freeze exactly where they are. That one frozen set of values, saved, is the model. Training is the name for the process; the model is what that process leaves behind.
Most frozen models don't keep growing on their own. The calculation that produces an answer doesn't change the dials. Some services, such as recommendations, do gather people's reactions to keep training running and swap in freshly adjusted models often. It's the same as barely learning anything new while riding a bike you already know how to ride. Changing what it can do means running training all over again — though there's also a separate approach where a finished model gets fed a little more new data just to shift its character.
More precisely
Training is an optimization process that repeatedly updates parameters in whatever direction shrinks the value of a loss function. That direction comes from calculating how sensitive the loss is to each parameter — that sensitivity is called the gradient, and moving a little in the opposite direction of the gradient is called gradient descent. In a deep neural network, the error measured at the last layer gets shared backward, layer by layer, to figure out each earlier layer's share of the blame — that procedure is backpropagation. One full pass through the entire set of prepared examples is called an epoch.
The bike analogy breaks down in places too. A person picks it up after a few dozen falls, while a model usually goes through millions to billions of examples and adjusts its dials thousands to hundreds of thousands of times or more. A person also judges for themselves what to do differently after each fall, while a model can only find its direction once something outside it hands over a ruler for how far off it was. And a person carries some of that balance over to a different vehicle, while a model often wobbles badly the moment it steps just slightly outside the range it trained on.
Try it yourself
Common misconceptions
It's easy to think training is storing knowledge, but actually it isn't keeping the examples themselves one by one — it's nudging a set of dials a little at a time by exactly how far off the answer was. A side effect is that a model can end up nearly memorizing some examples.
It's easy to assume training longer always makes things better, but actually if the same examples are repeated too many times, it can start memorizing what it's seen past a certain point, and get worse on anything new.
It's easy to think a service keeps training while you use it, but actually the dials don't change at the moment it's producing an answer. Depending on the service and its settings, though, conversations or records that users leave behind may be gathered up and used in a later round of training.
One-line summary
Training is watching examples and nudging a set of dials a tiny amount, an enormous number of times, by exactly how far the answer was off — and what's left behind once that process stops is the model.
Spotted an error or have a better analogy? Suggest an edit · Last updated 2026-10-18