After language models took over the world of artificial intelligence, researchers are turning to systems that build an internal representation of the environment, simulate the future, and learn through action. The goal is not just to formulate a convincing answer, but to understand how the world might change as a result of each decision.
The rapid progress of artificial intelligence over the past decade has largely been based on a seemingly simple principle: make the model bigger, feed it more data, and devote more computing power to training. Scaling studies have shown that the performance of language models improves predictably as the number of parameters, the training set, and the amount of computation are increased.
This approach has given rise to systems that can write programs, solve problems, analyze documents, and produce text and images. However, success in information-based tasks is not necessarily the same as the ability to operate in a changing world.
A robot that needs to pick up a cup, an agent navigating an unfamiliar environment, or a system that operates a device in a laboratory cannot settle for an answer that sounds reasonable. They must anticipate how an action will change the situation, detect an error in real time, and choose an alternative action.
This is where world models come into play.
What is a world model?
A world model is an internal representation of an environment and how it changes over time. The system tries to learn not only what objects are in the environment, but also how they move, how they react to each other, and what is expected to happen after a certain action.
Such a model might, for example, estimate how a robotic arm movement will affect an object placed on a table. Rather than immediately performing the action, the agent can examine several options within the internal model and choose the one that is more likely to succeed.
The model does not have to reproduce every detail in the world. Sometimes it is enough for it to represent important features: the object's location, direction of movement, distance from the target, and the chance of collision.
This can be compared to human planning. A person who wants to place a full glass on a shelf does not calculate the movement of every particle of liquid, but he does estimate the weight of the glass, the height of the shelf, and the possible result of moving too abruptly.
The difference between a language model and a world model
A large language model is typically trained to predict the next unit of information in a sequence. It takes text, an image, or other type of information and tries to produce the most likely continuation.
This way of training allows him to learn a huge amount of connections, facts, and patterns. It does not prevent him from building certain representations of the world, but the basic goal of his training is not necessarily to predict the physical result of an action.
A world model focuses on transitions between states. It takes a current state, a possible action, and sometimes a desired goal, and tries to predict the next state.
The distinction is not absolute. Language models can help with planning, andWorld models Can include language. The difference lies in the emphasis: one system learns primarily from the recorded sequence, while the other tries to learn how the environment responds to the action.
Genie 3 creates interactive environments
One of the main directions of research is the creation of simulated worlds in which agents artificial intelligence Can practice.
Google DeepMind presented the Genie 3, a world model capable of creating interactive environments from a verbal description. According to the company, the model creates a world that can be navigated in real time at 720p resolution and 24 frames per second.
The environment is not a fixed, pre-made video. It is created on the fly and responds to the direction the user or agent is facing. The model tries to maintain consistency of the objects and the environment over several minutes.
Such worlds could be used not only for creating games, but also as training grounds for autonomous agents. Instead of collecting all the real-world experience, the agent could be placed in thousands of environments, changing the conditions and seeing how it responds.
Access is still limited. Genie 3 can't maintain a consistent world for hours, doesn't always accurately recreate real-world locations, and sometimes struggles to display clear text within the environment.
SIMA 2 learns to operate within 3D worlds
Creating a virtual world is only half the battle. You also need an agent to operate within it.
PASS 2 Google DeepMind's AI is designed to receive language instructions and execute them in 3D environments. The agent sees the environment through the image displayed on the screen and operates using commands similar to a keyboard and mouse.
The first version of SIMA trained on hundreds of skills, such as navigation, opening a map, and interacting with objects. SIMA 2 incorporates Gemini capabilities, allowing it to interpret more complex goals, converse with the user, and monitor its progress.
The connection between Genie and SIMA demonstrates a possible division of roles: one model creates a dynamic world, while the other learns to operate within it. This allows for a training process in which both the tasks and the environment change.
Visualize ideas instead of every pixel
A different approach is being developed at Meta using the JEPA family—a predictive architecture based on shared representations.
Many video prediction systems attempt to create the entire future image, pixel by pixel. This requires a lot of computing resources and forces the model to deal with details that are not essential to the task.
V-JEPA 2 attempts to predict the future in a more abstract space. Instead of accurately reproducing every shadow, color, and texture, it learns representations of motion and the relationships between objects.
According to Meta, the model was first trained using self-learning from natural videos. It was then fed 62 hours of robot data from the DROID database. The combination allowed it to plan actions for a robotic arm in environments that did not appear in the training.
The tasks involved reaching for an object, grabbing it, and moving it to another location. The target was presented to the model as an image of the desired state, and the system attempted to plan a sequence of actions that would lead to it.
The main idea is that a large part of visual physics can be learned from ordinary videos, and only then can a smaller amount of valuable data from real robots be added.
GR00T connects vision, language and action
Nvidia Develops GR00T N1, a basic model for humanoid robots. The model receives visual information and language instructions and produces actions that can be transmitted to the robot's control system.
The training combined several types of data: videos from a human perspective, movements of real robots, trajectories from simulations, and synthetic data.
The combination is designed to address one of the key problems inרובוטיקה: Physical action data is much more difficult and expensive to collect than text or images. A robot needs to actually perform the action, and sometimes a human operator is also required to demonstrate it.
Simulations and synthetic data allow for expanding the training pool, but real-world testing is still needed. A small difference in friction, camera, lighting, or object shape can affect the success of the operation.
DreamZero envisions the world and robot action together
NVIDIA Research's DreamZero represents a direction known as the World Action Model.
Instead of just predicting how the video will look in the future, the system simultaneously learns the action the robot should perform. It connects a visual state, the robot's movement, and the expected state after the movement.
The model is based on a video system with 14 billion parameters, but the researchers adapted it for real-time operation. According to the project results, it is capable of implementing feedback control at a rate of seven times per second.
In robot experiments, the researchers reported a more than two-fold improvement in generalization to new tasks and environments compared to vision-language-action models used for comparison.
The intended benefit is not just performing a pre-learned task. The goal is for a robot to be able to deal with a different object, a new location, or an instruction that did not appear exactly in the training database.
Why is the physical world so difficult?
Text is constructed from a defined sequence of symbols. The physical world is continuous, noisy, and changing.
A camera may see the same object differently depending on the lighting and angle. A soft object responds differently than a hard object. The same movement may succeed one time and fail another time because of a small change in position.
A physical mistake can also be costly. A wrong answer from a chatbot can be corrected with new text. A bot's wrong action can break an object, damage equipment, or endanger a person.
Therefore, world models need not only to be accurate, but also to acknowledge uncertainty. A safe system needs to know when its prediction is unreliable and avoid risky action.
Will world models replace language models?
It is more likely that the two approaches will combine.
A language model is suitable for understanding instructions, explaining decisions, searching for information, and breaking down a complex task into steps. A world model is suitable for spatial prediction, planning movement, and evaluating the results of actions.
A future robot might use a language model to understand the request "set the table," and a world model to decide which object to start with, how to grasp it, and where to place it.
Even within a virtual environment, a language model can define a goal while the world model will examine possible paths to achieving it.
The challenge is to connect the systems without creating a chain where an error in one model is passed on to the next. This will require testing mechanisms, confidence measures, and continuous feedback from the environment.
Moving from watching to experiencing
The large language models have shown how far one can go by learning from the information that humans have already recorded.
World models add another layer: learning from change, action, and outcome. Instead of just asking "What is the likely outcome?", they also try to answer the question "What will happen if I act this way?"
Existing systems still lack a general understanding of the world. They operate in limited environments, make inaccurate predictions, and sometimes have difficulty transferring skills from one task to another.
However, the research on Genie 3, SIMA 2, V-JEPA 2, GR00T, and DreamZero points to a clear trend: the transition from artificial intelligence that analyzes information to artificial intelligence that tries to predict the results of its actions.
Questions and Answers
What is a world model in artificial intelligence?
A world model is an internal representation of an environment and how it changes. It allows a system to predict a future state based on the current state and possible action.
Is a world model a simulation?
Sometimes yes, but not always. A world model can create a full visual environment, like Genie 3, or operate in an abstract space of representations, like V-JEPA 2.
Why is a language model not enough to operate a robot?
A language model can understand instructions and suggest a plan, but a robot also needs to recognize distances, predict movement, handle touch, and respond to changes in real time.
How do world models train?
Training may include videos, simulations, human demonstrations, sensor data, and real robot trajectories. Sometimes, you start with a large amount of video and add action data later.
Will world models lead to general artificial intelligence?
This is still an open question. World models may improve planning, spatial understanding, and robotics, but there is currently no proof that they alone will be sufficient to create general artificial intelligence.
Primary sources
Scientific articles and publications
For the scientific article on the scaling laws of language models:
Scaling Laws for Neural Language Models
For Google DeepMind's scientific publication on Genie 3:
Genie 3: A New Frontier for World Models
For Google DeepMind's scientific publication on SIMA 2:
SIMA 2: An Agent That Plays, Reasons and Learns in Virtual 3D Worlds
For the article and Meta research project on V-JEPA 2:
V-JEPA 2: World Modeling, Understanding and Robot Planning
For NVIDIA's scientific paper on GR00T N1:
NVIDIA Isaac GR00T N1: An Open Foundation Model for Humanoid Robots
For the scientific article on DreamZero:
DreamZero: World Action Models Are Zero-Shot Policies
To the source: To the source
More on the subject on the science website
- More articles, fewer ideas? The danger of using artificial intelligence in science
- The AI Arms Race: Why Anthropic Wants to Hit the Brakes
- UN report: Artificial intelligence consumes not only electricity, but also water, land and metals
- Shifters Raises $10.2 Million to Develop Autonomous Ground Robotics
- Artificial intelligence accelerates the search for new chip materials
