AutoEnv: Automated Environments for Measuring Cross-Environment Agent Learning
Jiayi Zhang, Yiran Peng, Fanqi Kong, Yang Cheng, Yifan Wu, Zhaoyang Yu, Jinyu Xiang, Jianhao Ruan, Jinlin Wang, Maojia Song, HongZhang Liu, Xiangru Tang, Bang Liu, Chenglin Wu, Yuyu Luo
2025-11-25
Summary
This paper explores how well AI agents can learn to perform tasks not just in one environment, but across many different and changing environments, much like humans do. It identifies a gap in current AI research where agents usually get really good at one specific thing, but struggle when things change even slightly.
What's the problem?
Currently, AI agents are typically trained within a single, fixed environment. This means they don't learn general skills that can be applied to new situations. There's a lack of good tools to create a variety of environments for testing this kind of learning, and also no clear way to measure how well an agent is actually learning to adapt. Essentially, we don't have a good way to see if AI can truly learn *how* to learn across different worlds.
What's the solution?
The researchers created a system called AutoEnv that automatically generates a wide range of different environments by changing the rules, what the agent sees, and what rewards it gets. They used this to build a dataset of 36 environments, called AutoEnv-36, and tested how well different AI approaches could learn across them. They also came up with a way to break down the learning process into three steps: choosing the right learning method, improving the agent, and then testing it. They tried out eight different learning methods and saw how they performed as the number of environments increased.
Why it matters?
This work is important because it highlights how difficult it is for AI to generalize learning across different environments. It shows that simply using one learning method isn't enough, and that even choosing the best method for each environment becomes challenging as the number of environments grows. The tools they created, AutoEnv and AutoEnv-36, provide a standard way to test and improve AI's ability to learn and adapt, bringing us closer to AI that can handle the complexities of the real world.
Abstract
Humans naturally adapt to diverse environments by learning underlying rules across worlds with different dynamics, observations, and reward structures. In contrast, existing agents typically demonstrate improvements via self-evolving within a single domain, implicitly assuming a fixed environment distribution. Cross-environment learning has remained largely unmeasured: there is no standard collection of controllable, heterogeneous environments, nor a unified way to represent how agents learn. We address these gaps in two steps. First, we propose AutoEnv, an automated framework that treats environments as factorizable distributions over transitions, observations, and rewards, enabling low-cost (4.12 USD on average) generation of heterogeneous worlds. Using AutoEnv, we construct AutoEnv-36, a dataset of 36 environments with 358 validated levels, on which seven language models achieve 12-49% normalized reward, demonstrating the challenge of AutoEnv-36. Second, we formalize agent learning as a component-centric process driven by three stages of Selection, Optimization, and Evaluation applied to an improvable agent component. Using this formulation, we design eight learning methods and evaluate them on AutoEnv-36. Empirically, the gain of any single learning method quickly decrease as the number of environments increases, revealing that fixed learning methods do not scale across heterogeneous environments. Environment-adaptive selection of learning methods substantially improves performance but exhibits diminishing returns as the method space expands. These results highlight both the necessity and the current limitations of agent learning for scalable cross-environment generalization, and position AutoEnv and AutoEnv-36 as a testbed for studying cross-environment agent learning. The code is avaiable at https://github.com/FoundationAgents/AutoEnv.