SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds
Jiawei Ren, Yan Zhuang, Xiaokang Ye, Lingjun Mao, Xuhong He, Jianzhi Shen, Mrinaal Dogra, Yiming Liang, Ruixuan Zhang, Tianai Yue, Yiqing Yang, Eric Liu, Ryan Wu, Kevin Benavente, Rajiv Mandya Nagaraju, Muhammad Faayez, Xiyan Zhang, Dhruv Vivek Sharma, Xianrui Zhong, Ziqiao Ma, Tianmin Shu, Zhiting Hu
2025-12-03
Summary
This paper introduces SimWorld, a new computer simulation environment designed to help develop and test AI agents that can operate in realistic, complex situations like the real world.
What's the problem?
Current AI agents are really good at things like math and coding, but struggle when you put them in situations that require understanding the physical world and interacting with people. Existing simulations aren't helpful because they're too simple, often look like video games, and don't work well with the most advanced AI models like those based on large language models (LLMs). They lack the detail and flexibility needed to train AI to truly function in the real world.
What's the solution?
The researchers created SimWorld using Unreal Engine 5, a powerful game engine, to build a highly realistic and detailed simulation. SimWorld allows for environments to be created based on language prompts, meaning you can *tell* it what kind of world to build. It also lets AI agents 'see' and 'act' in the world using natural language, and it supports a wide range of tasks and scenarios. They tested SimWorld by having different AI models perform complex delivery tasks that required cooperation and competition between agents.
Why it matters?
SimWorld is important because it provides a platform for building and testing AI that can actually function in the real world, potentially leading to AI that can perform useful jobs or even run businesses autonomously. By making SimWorld open-source, the researchers hope it will become a standard tool for AI development across many different fields, accelerating progress towards more capable and practical AI.
Abstract
While LLM/VLM-powered AI agents have advanced rapidly in math, coding, and computer use, their applications in complex physical and social environments remain challenging. Building agents that can survive and thrive in the real world (for example, by autonomously earning income or running a business) requires massive-scale interaction, reasoning, training, and evaluation across diverse embodied scenarios. However, existing world simulators for such development fall short: they often rely on limited hand-crafted environments, simulate simplified game-like physics and social rules, and lack native support for LLM/VLM agents. We introduce SimWorld, a new simulator built on Unreal Engine 5, designed for developing and evaluating LLM/VLM agents in rich, real-world-like settings. SimWorld offers three core capabilities: (1) realistic, open-ended world simulation, including accurate physical and social dynamics and language-driven procedural environment generation; (2) a rich interface for LLM/VLM agents, with multimodal world inputs and open-vocabulary actions at varying levels of abstraction; and (3) diverse and extensible physical and social reasoning scenarios that are easily customizable by users. We demonstrate SimWorld by deploying frontier LLM agents (e.g., GPT-4o, Gemini-2.5-Flash, Claude-3.5, and DeepSeek-Prover-V2) on long-horizon multi-agent delivery tasks involving strategic cooperation and competition. The results reveal distinct reasoning patterns and limitations across models. We open-source SimWorld and hope it becomes a foundational platform for advancing real-world agent intelligence across disciplines: https://simworld.org.