Key points
- A bias term is the default amount that always gets added, no matter what the input is. Unlike a weight, it isn't multiplied by anything — it's just tacked on once.
- What it actually does is shift the threshold. A large default value means the neuron fires even with weak input; a small one means it takes a strong input to fire at all.
- Without a bias term, the sum a neuron computes must be exactly zero whenever every input is zero. That narrows the range of judgments it can express.
- Every neuron gets one. There are far fewer of them than weights, but dropping them with nothing to take over their job means the boundary can't shift, which narrows what the model can express.
- The name is "bias," but it's not the same word as the bias that means discrimination or skewed data. It's just a constant tacked onto the calculation.
The analogy
A taxi meter starts from a fixed amount the moment you sit down and close the door. Go one block, or sit stuck at a red light without moving an inch, and that flat starting fee still applies. Distance gets added on top of it.
That flat fee is a dial that shifts the whole fare up or down. Raise the flat fee for a late-night surcharge, and the same distance now starts from a higher rung on the ladder. If you've set yourself a budget of, say, twenty dollars for the ride, on a night with a high flat fee you'll hit that budget after going a short distance, and on a night with a low one you'll need to go much farther to hit it. The per-block rate never changed — only where the limit falls shifted back and forth.
The bias term is exactly this flat fee. If the weight multiplied against an input is the per-block rate, the bias term is the starting amount that's there even before you've gone anywhere. It's what pulls the point where a neuron fires closer or pushes it farther away.
In detail
It adds, it never multiplies
A neuron multiplies every incoming value by its weight, sums them all up, and then, at the very end, adds the bias term once. Unlike every other number in the calculation, the bias term has no matching input. It's always there on its own.
That means a neuron can still produce a nonzero value even when every input is zero. Without a bias term, the summed value would have to be exactly zero whenever nothing comes in at all, and the shapes of judgment a neuron could express would be limited to ones that pass through the origin.
The dial that shifts the threshold back and forth
An activation function has a fixed line where it turns on. That line itself never moves — the function sets it once and it stays put. Instead, the bias term shifts the incoming value up or down as a whole, making it easier or harder to cross that fixed line.
Set the bias term high, and that neuron becomes a sensitive detector that fires at almost any signal. Set it strongly negative, and it becomes a picky detector that barely fires at all. Because this default value differs from neuron to neuron even within the same layer, some end up tuned to common signals and others to rare ones.
Without it, the range of judgments narrows
A model still runs if you strip the bias term out. But every decision boundary is now forced to pass through the origin. If the data sits far from the origin, there's no way to pull the boundary off the origin, so no amount of training may be enough to split the data properly. Stacking layers doesn't fix it if there isn't a single bias term anywhere, since with common activation functions the constraint stays. Pulling the boundary off the origin takes a bias term or something that does its job.
Adding a single bias term back in makes the problem disappear. Now the boundary can shift as a whole, so the weights only have to worry about direction and slope. There are only as many bias terms as there are neurons — a tiny slice of a model's total parameters — yet nearly every model keeps them anyway, precisely for this reason.
It gets tuned right alongside training
A bias term is trained exactly like a weight is. It usually starts at zero or something close to it, and gets nudged little by little based on how far off the prediction was. It isn't a setting a person picks in advance — it's a value the model finds for itself.
Open up a trained model and you'll see the bias terms differ neuron by neuron. A large negative one means that neuron leans toward staying off for most inputs. A positive one means it leans toward switching on easily. Whether it actually fires depends on the incoming values too.
Sometimes there's no room left for it
When a network already uses a separate device to keep the scale of values even across a layer, the bias term is sometimes dropped entirely. That device is already doing the job of shifting values, so adding a bias term on top of it would just be doing the same work twice.
In large language models, some layers drop the bias term outright. Reports show barely any difference in performance, and training of large models turned out more stable. A default value isn't always essential — where something else can cover the role, the bias term can step aside.
More precisely
A bias term is a constant added to a neuron's computation. In the language of a straight line, the weight sets the slope and the bias term sets where the line crosses the vertical axis, which is why it's also called an intercept. It gets adjusted alongside the weights during training, one per neuron.
The taxi-fare comparison breaks down in one place. A flat fee is always positive, but a bias term is very often negative. A negative bias term is actually a common tool for keeping a neuron mostly silent and reactive only to strong signals. And a fare is set by policy, decided by a person, while a bias term is set automatically by the data.
There's also a naming confusion worth flagging. The problem where training data skews one way and produces answers unfair to some group is also called "bias" in everyday English, but that's a story about data and society. The bias term discussed here is just a single number tacked onto a formula, unrelated in meaning even though it happens to share the same word — context is the only way to tell them apart.
Try it yourself
Common misconceptions
It's easy to think a bias term is the same thing as the bias that means data-driven discrimination, but actually it's just a constant added to a computation — the two only share a name.
It's easy to assume a bias term barely matters, but actually it's a dial that shifts a decision boundary, so unless something else covers that job, dropping it narrows what the model can express.
It's easy to think a bias term gets multiplied by an input too, but actually it has no matching input at all — it's just added once at the end.
One-line summary
A bias term is the default amount that always gets added regardless of the input, and its job is to shift the point where a neuron fires back and forth.
Spotted an error or have a better analogy? Suggest an edit · Last updated 2026-10-17