Imagine a virtual world entirely driven by neural networks - no traditional game engine, not purely hand-written physics, but a combination of structured priors and learned dynamics.
The model autonomously learns to understand how the world evolves and generates every frame in real time. Now extend this from single-player to multiplayer: multiple players coexist in the same world, cooperating or competing, with each player's view rendered in real time by the model from their unique perspective.
This is the Multi-Agent World Model we are building.
We chose CS:GO as our starting point. Compared to simpler environments like DOOM or GoldenEye used in prior work, CS:GO features more complex 3D scenes, richer interaction mechanics, and higher visual fidelity - providing a far more challenging testbed for world model research.
Core Challenges
Multi-View Consistency
In multi-agent scenarios, different players observe the same world from their own perspectives. At the current stage, we focus on the most fundamental form of consistency: when other players move, turn, or act in the world, these changes should appear correctly in the ego player's view with the correct position, scale, and motion trajectory. In other words, the model needs to keep the same player spatially and behaviorally consistent across viewpoints, which serves as the foundation for modeling more complex multi-agent interactions.
World State Prediction
We believe a good world model should not only generate "plausible-looking frames" but truly predict the impact of each interaction on the entire world state. Inferring global changes in a complex system from partial observations of a single viewpoint is extremely difficult. We therefore focus on predicting the evolution of the entire system state in a relatively controlled environment.
Generalization
A valuable world model should not be confined to a single scenario. We aim for the world knowledge learned by the model to transfer across different maps, visual styles, and even entirely different virtual environments. As a general-purpose world simulator, scalability is the key to broader applications.
Real-time Generation
For world models to deliver practical value, real-time performance is essential. In embodied AI, high-performance multi-agent world models can provide efficient simulated training environments for complex scenarios like multi-robot cooperation. A real-time multiplayer world model may also become a prototype for next-generation game engines.
Technical Approach
A central challenge in multi-agent world modeling is how to handle both world-state evolution and multi-view visual generation. Related works such as MultiGen and Agora-1 have highlighted the value of decoupling state modeling from visual generation: first evolving a shared world state, then rendering observations from different viewpoints. Our work continues along this problem setting, focusing on how to build a closed-loop system that connects state prediction, action conditioning, and neural rendering in more complex, high-fidelity 3D multi-agent scenes.
Unlike the relatively simplified environments used in these studies, we choose CS:GO as our starting point. CS:GO contains detailed 3D geometry, more complex player movement and complex occlusion between players, and higher visual fidelity requirements. This means the model must not only predict how player states evolve, but also render other players in the ego view with correct position, scale, and action changes, posing stronger demands on both state modeling and conditional rendering.
In our method, we decouple shared state evolution from multi-view visual generation, separating what happened in the world from what a particular player sees. We employ a shared world state prediction model to update key entities, spatial structure, and interaction relationships while preserving continuous memory through time. Every player has their own conditional renderer providing its visual observation according to the player input and the world state.
This design keeps players' views consistent while supporting fluid interaction. And the rendering cost grows approximately linearly with the number of players while preserving full consistency.
Outlook
We believe multi-agent world model research will open doors in multiple directions:
Embodied Intelligence
Efficient multi-agent world simulators can provide realistic and scalable training environments for multi-robot cooperation, multi-agent reinforcement learning, and more.
Interactive Content Creation
World models could become the next generation of content creation engines, dynamically generating rich interactive scenarios for gaming, virtual social spaces, and beyond.
Foundation for Advanced Intelligence
Understanding and simulating a world of multi-agent interactions is an important step toward building more general and powerful artificial intelligence.
We will continue to explore the core challenges facing multi-agent world models and push the boundaries of this research direction.