Key points
- LoRA (Low-Rank Adaptation) locks the main body in place and trains only a small add-on.
- What it learns is a tiny sliver of the whole, so training is light, and the result is a single file far smaller than the main body (often tens to thousands of times smaller).
- The add-on snaps on when needed and comes off when it isn't. One main body can swap between several add-ons, or layer more than one at once.
- It's the usual way to package a particular art style or character in image generation, and a common way to fit a language model to a company's tone.
- It's weak at teaching a field the main body barely knows from scratch. It's closer to leaning what's already there in a certain direction.
The analogy
An optician doesn't fix your eyes when your vision is off. The eyes stay exactly as they are — a thin lens gets held up in front of them instead. Sitting in the exam chair, you try lens after lens until the letters snap into focus, and glasses get made to that exact prescription. Nothing about how your eyes work has changed, yet what you see is entirely different. And that lens is small and light. Keep one pair for driving and another for reading, swap between them as the moment calls for it, and your eyes stay exactly the same the whole time. Don't need them — take them off, and that's the end of it. LoRA leaves a large main body untouched and holds a thin lens up in front of it instead.
In detail
The main body stays locked
Retuning a whole model shifts every internal value a little. LoRA locks those values entirely instead. Not a single value in the main body changes while training runs.
Instead, a small add-on gets attached at chosen spots within each layer, and only that add-on gets trained. During a calculation, the value the add-on produces gets added onto the value the main body produces before passing along to the next step. Unlike the lens, they don't take turns: the main body and the add-on receive the same input side by side, and their outputs get added together.
Why the add-on can stay so small
Here's the question this naturally raises: if the main body holds billions of values, what can an add-on with only a few million possibly accomplish?
The key is that what needs to change isn't the values themselves — it's how much they need to shift. The shift needed to fit a certain tone or art style tends to be simple in structure. It boils down to a handful of directions, with everything else sitting close to zero. Something with a simple structure like that can be captured with nothing more than two thin layers stacked together. That's why shrinking the add-on to far smaller than the original spot still captures nearly all of the needed shift.
How thin to make it is a setting a person chooses. Thinner means a smaller, faster file that can hold less of a shift; thicker means the reverse.
Swapping and stacking
Retuning the whole model leaves behind a brand-new model the same size as the main body, every single time. Ten uses means ten large files piling up. LoRA leaves you with one main body and ten small add-on files. Storage space and the hassle of moving files around are a different order of magnitude.
It's flexible to use, too. Swap in one art style now, another later, or layer two add-ons together and blend them. You can even dial how strongly each one gets applied. Downloading a style file for an image generation tool and dropping it in is exactly this.
What "light" actually means
Training gets easier on several fronts at once. Fewer values to correct means fewer gradients and less optimizer state (the training bookkeeping kept for each value) to carry too. So models that would be out of reach to retune fully become workable to adapt on personal hardware.
The result stays light as well. If the main body runs tens of gigabytes, the add-on commonly drops to tens of megabytes. That lightness is exactly what made a whole culture of sharing style files possible.
What it can't do
A lens only shapes the light your eyes already receive. It can't make visible something that never reached your eyes in the first place. LoRA is similar. It's a weak fit for teaching a field the main body barely knows from scratch.
There's a limit when the shift needed is very large, too. The kind of shift it can hold has a fixed shape, so pulling the main body toward a character entirely different from what it already knows still calls for retuning the whole thing. When there's plenty of data and every last bit of performance matters, full retuning remains the choice.
More precisely
LoRA represents the shift to be added to a layer's weights as the product of two small matrices, and trains only those two matrices, instead of updating the weights directly. The name comes from the observation that this shift can be captured well enough at low rank. The value that sets how thin to make it is called the rank, and there's a separate value that sets how strongly it gets applied.
The analogy breaks down in places. A pair of glasses sits as one lens in front of the eyes, but LoRA attaches in several places within each layer, and which spots to attach to is itself a choice to make. There's another gap: take glasses off and you're back to normal, but a trained add-on can be merged straight into the main body. Merging it in speeds up calculation a little, at the cost of losing the ability to swap it in and out. And an add-on has to match its main body — drop it onto a different one, and it either goes strangely off, or doesn't attach at all.
Try it yourself
Common misconceptions
It's easy to think LoRA is something entirely separate from fine-tuning, but actually it's one branch among several ways of adapting a model to a purpose — the branch that leaves the main body untouched.
It's easy to think one add-on file works anywhere, but actually it has to match the main body it was built on to attach properly.
It's easy to think being light means performance drops to match, but actually for matching tone or art style, it often lands close to what full retuning achieves.
One-line summary
LoRA locks a large main body in place and trains just the small add-ons attached throughout its layers, so they can be swapped in and taken off whenever needed.
Spotted an error or have a better analogy? Suggest an edit · Last updated 2026-10-17