Pruning
Cutting away unused connections to make a model lighter
- Pruning removes connections inside a model that are barely ever used. It separates what stays from what goes.
- A trained model ends up with an enormous number of connections whose value sits near zero. They're doing almost nothing, so cutting them barely changes the answer.
- Nobody cuts a huge chunk all at once. Cut a little, then retrain, over and over — that's what keeps performance from dropping too much.
- There's cutting scattered connections here and there, and there's removing whole rows or blocks at once. It's usually the wholesale kind that actually speeds things up.
- It's a different move from quantization, which coarsens the scale numbers are written on. Pruning reduces the count itself.
Contents
1The analogy
A food stall's menu board lists about eighty items. But tally up a year's worth of orders and half of them turn out to have never been ordered even once — names still hanging on the board while the ingredients quietly go bad in the fridge.
So the never-ordered items get struck from the board. Ingredient space opens up, the kitchen workflow gets shorter, and there's less fumbling when an order comes in. What customers actually get barely changes — they were only ever ordering what they always ordered anyway.
Cut forty items at once, though, and trouble follows. Even the odd dish someone orders once in a while disappears, and regulars start walking away. Safer to cut about ten, watch for a month, then cut ten more. Pruning is this same process of trimming away what isn't used.
2In detail
Why so many unused connections pile up
A model isn't built to be exactly the right size from the start. It's built generously oversized on purpose and then trained — going in big tends to make training go better. So once training wraps up, what's left is a mix: connections that genuinely carry weight toward the answer, and connections whose value just hovers near zero.
A connection near zero contributes almost nothing to the result no matter what it's multiplied against. And yet it still takes up its spot in the calculation and still triggers a multiplication every single time. It's not unusual for connections like this to make up more than half the total.
Pruning picks out these connections and cuts them. What's left behind at each cut spot is a zero — a marker the calculation can learn to simply skip over.
How to pick what to cut
The most common yardstick is simply the size of the value. Line everything up from closest to zero and cut from the bottom. It takes little effort and works reasonably well.
Going by value alone misses some things, though. A small value sitting at exactly the wrong bottleneck can bring everything downstream crashing when it's cut. So sometimes the actual effect of removing a given connection on the final answer gets measured directly, and choices get made from there. How much can safely be cut also differs layer by layer — earlier layers generally have little room to spare, while later ones tend to have more.
Cut a little, then retrain
Cut a large chunk all at once and the answer noticeably gets worse. So instead, cut a little, then retrain briefly on what's left so it can fill in the gap. Repeat this two-step cycle several times over.
Even cutting the same total amount, this approach loses far less performance. The connections that remain gradually pick up a share of what the removed ones used to do. Cut everything up front with no retraining at all, on the other hand, and the answer degrades in rough proportion to how much got cut.
Scattered pruning versus wholesale pruning
Cut connections one at a time and what's left ends up scattered all across the table. The table's shape stays the same even with plenty of cells gone empty, so an ordinary processor still scans the whole table regardless. Storage shrinks, but speed often doesn't pick up nearly as much as hoped.
That's why wholesale removal — cutting entire rows or blocks at once — often gets used alongside it. Removing one whole unused neuron, or one whole filter in an image model that barely ever fires, are examples. The table genuinely shrinks this way, so the computation genuinely shrinks too. In exchange, performance wobbles more per cut, so this kind of cutting has to be done more cautiously.
How it differs from quantization
Both make a model lighter, but they touch different things. Quantization doesn't throw away a single number — it only coarsens the scale those numbers are written on. Pruning leaves the scale untouched and instead reduces how many numbers there are.
In menu-board terms: quantization is leaving every item on the board and just rounding all the prices to the nearest dollar; pruning is striking the never-ordered items off the board entirely. Neither gets in the other's way, so they're often used together — cut what's unused first, then coarsen the scale of what's left.
3More precisely
Pruning removes low-contribution weights or neurons from a trained network to make it sparse. It splits into unstructured pruning, which removes individual weights, and structured pruning, which removes entire channels or layers — the latter is more likely to translate into a real speedup. There's also a widely discussed idea that a well-performing smaller skeleton already sits hidden inside a large model from the start.
The analogy breaks down in a place too. An item struck from the menu board is genuinely gone — customers can't order it. A connection cut in the scattered way, though, is still sitting there as a zero inside the table, so squeezing storage down for real needs a separate cleanup step. And a food stall judges popularity by order count, while a model's connections get judged by how much they shake the result, not how often they fire — some connections that almost never activate turn out to be decisive on the rare occasion they do. Whichever way you cut, run the evaluation again afterward to check what changed.
How much can safely be cut also isn't fixed in advance. It depends on the task, the training data, and how much of a quality drop is acceptable, so the same model pruned for two different purposes can end up at two very different final sizes.
4Try it yourself
5Common misconceptions
It's easy to think pruning just shrinks a model carelessly, but actually it's a careful process — picking the lowest-contributing connections first, cutting a little, and retraining to recover.
It's easy to think removing connections speeds things up immediately, but actually cutting them scattered leaves the table's shape unchanged, so speed barely improves.
It's easy to think pruning and quantization are the same thing, but actually one reduces the count of numbers and the other coarsens the scale they're written on — two different moves entirely.
7One-line summary
In shortPruning picks out connections that barely contribute to the answer, cuts them away, and retrains what's left to make a model lighter.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02