Regression
Naming a single number somewhere on a continuous scale
- A regression's answer comes out as a single number on a continuous scale — things like price, weight, or time, where values in between make sense.
- If the answer is a category, that's classification; if the answer is a number, that's regression. Whether an in-between value makes sense is what tells the two apart.
- A model doesn't memorize values. It finds a relationship, where the answer moves as the input moves, and uses that relationship to produce a new value.
- Regression's grade isn't right-or-wrong. It's measured by how far off the answer landed. That gap is what steers the training itself.
- Feed it an input outside the range it has seen, and it answers by just stretching the scale further out — which is exactly why it can go badly wrong there.
Contents
1The analogy
Pick up a bag of potatoes at a farm stand and the seller sets it on the scale. They read where the needle stops and call out a price — this much for 900 grams, a bit more once it passes a kilo. The price isn't decided by picking a whole sack — it's decided by where the needle lands on the scale. Regression is exactly this: pointing to a single number somewhere on a continuous scale.
The seller hasn't memorized a price for every possible weight. It's a feel built up from years of selling — heavier means the price climbs, and a bit more gets added if the potatoes are unusually large. That feel is why they can still call out a price for a weight they've never seen before.
Haggle a little, and how accurate that feel really is comes out. If the price the seller calls keeps drifting from the price that actually gets paid, they start nudging their called price from then on.
2In detail
The answer sits on a scale
What separates regression from classification is the shape of the answer. Sorting mail into Seoul-bound and Busan-bound has no in-between value. There's no such thing as a sack that's half Seoul and half Busan. The price of potatoes is the opposite — there's an in-between anywhere you look. Between 3,200 won and 3,300 won sits 3,250 won, and that value makes just as much sense.
If an in-between value makes sense, it's usually regression; if it doesn't, it's usually classification. A home's price, the time left until a delivery arrives, tomorrow's temperature — all regression. Whether an email is spam, whether a photo shows a dog or a cat — those are classification.
The same problem can even land on either side depending on how you frame the question. Ask for a test score as a number and it's regression; ask whether it's a pass or a fail and it's classification. What gets asked is a choice a person makes.
It finds a relationship instead of memorizing values
What a regression model does is set up a relationship between the input and the answer — how much the price climbs for every extra 100 grams, how much extra gets added when the potatoes run large. With just this relationship in hand, it can call out a price even for a weight that never showed up on the scale before.
Setting up that relationship means weighing several clues at once. Weight, size, season, and origin all nudge the price up or down a little. Some clues push hard, others barely at all, and finding that strength from the data is what training is.
The relationship doesn't have to be a straight line either. It can handle a curved relationship too — price climbing as weight goes up, then leveling off once it crosses into wholesale territory. Just be careful: give it too much room to curve, and it starts tracing the wobble in the data itself.
The size of the miss is the compass
Classification grades itself as right or wrong, but regression can't be counted that way. Calling a 3,200-won bag 3,190 won and calling it 8,000 won are both wrong, but they're nowhere near equally wrong. That's why regression uses the gap between the called value and the real one as its grade.
Gather up many of these gaps and average them, and you learn how far off the model tends to be. Add the gaps up as they are and an overcall and an undercall cancel each other out, so the gap usually gets squared first, then added — a way of making a big miss sting harder.
This value is a report card and a compass together. Which direction it was off tells the relationship which way to shift. It's the same as a seller who kept calling too high a price starting to call a bit lower from then on.
Step outside the range it's seen, and the scale just stretches
The place regression needs the most care is a range that never showed up in the data. Ask a seller who's only ever sold bags between 100 grams and 2 kilos to price a 20-kilogram sack, and they'll just stretch out their usual feel and call a number. But that range actually runs on a separate wholesale price, and the feel doesn't hold up there.
A model does the exact same thing. It's excellent inside the range it saw in training, and the moment it steps outside that range, it just extends the relationship in a straight line anyway. That's how it ends up handing back an age that's negative, or a price that's less than zero, without so much as a flinch.
If you already know what range values should fall in, it's worth fencing the answer in ahead of time. And don't look at a predicted value alone — check whether it came from a part of the range where the data was thick or thin.
3More precisely
Regression is the problem of taking an input and producing a single continuous number. Because it trains on examples that already carry the correct answer, it belongs to supervised learning alongside classification — the difference is that the answer is a real number instead of a category. The most common ruler for grading it squares the gap and averages it; when you want a big error to sting less, the absolute value of the gap gets used instead. Because exactly how far off, and in which direction, an answer landed becomes the training signal directly, regression is also the clearest problem shape at spelling out exactly where training should head next.
The analogy breaks down in one place. The farm stand seller knows the reasons behind a price — cost, going rate — but a regression model doesn't know why. It only follows whatever moved together in the data. So it learns a reversed relationship just as readily. A record showing that umbrella sales climb on days it rains a lot can still produce a regression, but selling more umbrellas won't make it rain. The name regression itself comes from a statistical term meaning "reverting," and has nothing to do with reversing anything.
4Try it yourself
5Common misconceptions
It's easy to think regression means drawing a straight line, but actually it can handle curved relationships too, and weigh several clues at once.
It's easy to think it's a harder problem than classification, but actually only the shape of the answer differs, and the same problem can often be framed as either.
It's easy to think answering with a number makes it more precise, but actually stepping outside a range it's seen just stretches the scale further out, which is exactly where it can miss by the widest margin.
7One-line summary
In shortRegression is a prediction that names a single number on a continuous scale rather than a category, and how far off that number lands is both its grade and its compass for training.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02