World Model
A model of how the world works that you can run before acting
- A World Model is a model that holds inside it how the world tends to work. Feed it "do this" and it answers "then this happens next."
- Its use is trying things out before you actually do them. You test several moves inside it and pick one, without ever moving in the real world.
- Learning by trial and error in reality is slow, costly, and risky. Inside a model you can run thousands of tries fast, cheap, and safely.
- The model is only as good as it matches reality — off by that much. Chain several steps together and the gap snowballs.
- Lately it's meeting video generation head-on, showing up as something that generates the next frame every time you act.
Contents
1The analogy
Before planting straight into a field, growers bring seedlings into a greenhouse first. The glass house has soil, light, and air close to what's outside, so you can see how a seed will grow without waiting out an entire winter. You can water it more, water it less, and move it around, all without wasting a single real plot of field.
Everything hinges on how closely the glass house matches the outside. If it's always warm inside with never a breath of wind, whatever thrives there will collapse the moment it goes outside. A greenhouse is only worth anything when it holds a plausible version of what's actually out there.
And a greenhouse is most trustworthy for seeing one season ahead — ask it to predict several years out, and it starts drifting, because small differences grow as they compound.
2In detail
A model that holds what comes next
What a World Model does fits in one sentence: it takes the current state and the action just taken, and produces the next state. Push a cup and it slides; push harder and it falls off the edge; when it falls, it makes a sound. That's the chain of cause and effect it holds inside.
Nobody writes the rules into this model by hand. It's shown a huge pile of records of experiencing the world and made to guess the next moment, over and over, letting the model take shape on its own. Guessing the next moment well requires knowing what affects what, so that chain of cause and effect gets absorbed while the guessing repeats.
What ends up inside isn't a physics formula — it's closer to a much fuzzier sense of things. There's no formula, but it can still guess reasonably well how far something rolls for a given push.
Why run it in advance
Learning by trial and error only tells you the result once you've actually done it. Inside a game on a screen, repeating something tens of thousands of times is fine — but with a real machine, one mistake can break a part, and every single try can take minutes.
A model gets rid of that burden. Inside it, you can unfold several branches — go left, slow down, wait — and only carry out whichever one looks best in reality. That's attaching hundreds of imagined tries to every single real one, so learning happens with far less real experience.
Safety is a big part of it too. The more dangerous a situation is, the harder it is to let something actually experience it — but inside a model, a collision costs nothing. It also lets you deliberately manufacture rare, hard-to-encounter situations for practice.
What gets kept, what gets thrown away
Trying to predict a whole screen frame by frame is too much to handle. So the observation usually gets compressed into a much smaller representation, and the next step gets predicted from that small representation instead. Things like where something is and how fast it's moving get kept; things like wallpaper patterns or shadows get thrown away.
What gets thrown away decides whether this works. Throw away something that turns out to matter later, and that part becomes unpredictable. It genuinely happens that a small marker in the corner of the screen was the decisive clue, and it vanishes during compression.
Compressing things down also has a real upside. Prediction gets lighter, so imagination can run many times, very fast. Looking ahead thousands of times faster than real time becomes possible because of exactly this.
The gap piles up
One step ahead, it guesses well. The trouble starts when that result gets fed back in to chain a second step, a third step. The small error from the first step becomes the next input, and another error stacks on top of that. By around the tenth step, the world inside the model can end up looking nothing like reality.
That's why imagining is usually kept short. Look ahead just a few steps, take one real step, reset the state using the real result, and look ahead a few more steps. Real observation keeps catching the drift before it runs away.
Some setups also have the model report how confident it is in its own prediction. Wherever the model isn't confident, the system is built not to trust the imagined result and to try it for real instead.
Where it meets video generation
Lately, a version that generates the next frame directly as an image has been showing up. Press an arrow key and the matching next scene gets generated; press again and the scene after that gets generated — the screen itself is the prediction.
It's worth flagging that looking plausible and being useful aren't the same thing. Even if it looks smooth to the eye, if it doesn't respond consistently to input, or if going back to a spot you just passed through shows something different, it's useless for planning. The ability to generate a good-looking frame and the ability to hold onto the world's continuity need to be judged separately.
3More precisely
World models grew out of approaches that learn a model of the environment and then plan or learn a policy inside it. A common structure compresses observations into a latent representation and predicts both the next state and the reward to come, from within that latent space. Planning works by rolling several candidate actions out to the end inside the imagination and picking whichever branch comes out best.
The greenhouse analogy breaks down in places. A person designs a greenhouse, so you can check its blueprint to see what it's meant to imitate; a World Model builds itself from records, and it's hard to tell what it captured even by looking inside. Real plants grow in a greenhouse, but there's nothing real inside the model — only a guess about the next observation — so wherever the guess is wrong, the world quietly drifts off course. And a greenhouse runs at the same pace as the outside, while a model can run far faster than real time, and rewind the same situation to try again as many times as you like.
4Try it yourself
5Common misconceptions
It's easy to think guessing what comes next well means understanding the world, but actually stringing together familiar sequences well is different from knowing the underlying reasons, and it falls apart in situations never encountered before.
It's easy to think a plausible-looking video counts as a World Model, but actually whether it responds consistently to input and holds onto places already passed through needs to be checked separately.
It's easy to think doing well inside the model means doing well in reality, but actually performance drops by exactly the gap between the model and reality, which real attempts have to keep correcting for.
7One-line summary
In shortA World Model is a greenhouse that imitates the outside, letting you run through things many times inside before trying them for real — but the result drifts off by exactly as much as the greenhouse drifts from the outside.
Spotted an error or have a better analogy? Suggest an edit · Last updated2026-09-02