What is a world model? How it differs from video generation
A world model predicts how an environment changes as you act in it. What that means, how it differs from video generation, what people use it for, and where it falls short.
A world model is a model that predicts how an environment changes over time, given what happened before and what actions are taken. In generative video, that means a model that produces the next view of a scene as you move through it or act in it, instead of producing a clip you only watch.
This guide explains the idea, how it differs from ordinary video generation, what people build with it and what to expect.
The short definition
Give a world model a starting observation, such as an image, and an action, such as "move forward" or "turn left", and it produces what the world looks like next. Give it the next action and it continues. The model has no fixed script: the path is decided by the actions you feed it, in real time.
World model versus video generation
| Video generation | World model | |
|---|---|---|
| Input | A prompt, sometimes an image | A starting state plus a stream of actions |
| Output | A finished clip | An unfolding stream you steer |
| Who decides what happens next | The model, from the prompt | You or an agent, action by action |
| Length | Fixed per request | Open-ended |
| Control | Before generation | During generation |
| Typical use | Content creation | Interaction, simulation, exploration |
The line blurs in practice. Many realtime video models accept prompt changes while they run, and some world models also take a text prompt. What makes a model a world model is that actions condition what it predicts next.
Three kinds you will meet
- Navigable worlds. You hold W, A, S or D and look around inside a scene generated from a single image. LingBot-World, DreamX-World and ABot-World are examples, all hosted on Windflow. Matrix-Game 4.0 takes a different route: a real game runs the world and the model only redraws it. See navigable world models for games.
- Simulators for agents and robots. A world model can stand in for a physical or virtual environment so that a policy can be trained or tested against it. Odyssey, for instance, describes Odyssey-3 as a model that predicts how objects move and interact so developers can simulate environments and train policies; see this week's radar.
- Action-conditioned video models. Models trained with explicit action inputs, from keyboard keys to robot joint commands, so that the video reflects the commanded behaviour.
What people use them for
- Prototyping and previsualisation: explore a scene concept without building assets.
- Interactive experiences: demos, installations and prototypes where the novelty is that the world responds.
- Research: studying action-conditioned generation, long-horizon consistency and embodied agents.
- Simulation for training: generating environments in which to test or train agents.
Where they fall short
- They are not game engines. There is no persistent geometry, physics or rule system. What you saw behind you is regenerated when you turn back.
- They drift. Over long runs, identity, colour and layout wander because each chunk inherits earlier errors.
- Memory is limited. A rolling cache forgets old frames, so a world can change when you leave and return.
- Controls are quantised. Actions take effect at generation-block boundaries, so response is quick but not frame-exact.
- Output varies. The same seed image and controls do not guarantee the same world every time.
Plan products around those limits: short sessions, a deliberate reset and a visual style that tolerates a world that evolves.
Running a world model in real time
A world model is only useful interactively if it runs faster than playback and delivers frames with low delay. That needs a fast generation loop and a streaming layer; see how realtime video models work. Windflow hosts four navigable world models behind one session API, billed per billable second from $0.0023.
FAQ
What is a world model?
A world model predicts how an environment changes over time given its history and the actions taken. In generative video it produces the next view of a scene as you move or act in it, rather than a fixed clip.
What is the difference between a world model and video generation?
Video generation turns a prompt into a finished clip. A world model turns a starting state and a stream of actions into an unfolding scene, so what happens next depends on what you do.
Are world models open source?
Several are. LingBot-World, Matrix-Game, DreamX-World and ABot-World publish code and weights; each has its own licence, linked from its model page.
Can a world model replace a game engine?
Not for games that need exact geometry, physics or rules. They suit prototypes, previsualisation and experiences where a world that regenerates is acceptable.
Can I run a world model live in a browser?
Yes, through a hosted session: the model runs on a GPU and streams frames to the browser while you send controls. See add live AI video to your app.