FLEX: Continuous Agent Evolution via Forward Learning from Experience
Zhicheng Cai, Xinyuan Guo, Yu Pei, JiangTao Feng, Jiangjie Chen, Ya-Qin Zhang, Wei-Ying Ma, Mingxuan Wang, Hao Zhou
2025-11-11
Summary
This paper introduces a new way for AI agents, powered by large language models, to learn and improve *after* they've been initially trained, much like how humans learn from experience.
What's the problem?
Currently, AI agents using large language models are pretty good at tasks they're trained for, but they don't get better with practice or new experiences once they're deployed in the real world. They're essentially 'stuck' with the knowledge they had at the time of training, and can't adapt and grow over time.
What's the solution?
The researchers developed a system called FLEX, which allows these AI agents to continuously learn from their successes and failures without needing to adjust the core large language model itself. FLEX works by creating a sort of 'memory' of past experiences – a structured library of what worked and what didn't – and using that memory to guide future decisions. This is done without using traditional gradient-based learning, making it more efficient and scalable. They tested FLEX on tasks like solving math problems, designing chemical reactions, and predicting protein structures.
Why it matters?
This research is important because it's a step towards creating AI agents that can truly learn and evolve over time, becoming more capable and adaptable as they interact with the world. The fact that experiences can be 'inherited' by other agents also suggests a path towards building more robust and efficient AI systems, and shows a predictable pattern of improvement as the agent gains more experience.
Abstract
Autonomous agents driven by Large Language Models (LLMs) have revolutionized reasoning and problem-solving but remain static after training, unable to grow with experience as intelligent beings do during deployment. We introduce Forward Learning with EXperience (FLEX), a gradient-free learning paradigm that enables LLM agents to continuously evolve through accumulated experience. Specifically, FLEX cultivates scalable and inheritable evolution by constructing a structured experience library through continual reflection on successes and failures during interaction with the environment. FLEX delivers substantial improvements on mathematical reasoning, chemical retrosynthesis, and protein fitness prediction (up to 23% on AIME25, 10% on USPTO50k, and 14% on ProteinGym). We further identify a clear scaling law of experiential growth and the phenomenon of experience inheritance across agents, marking a step toward scalable and inheritable continuous agent evolution. Project Page: https://flex-gensi-thuair.github.io.