Skill Expansion and Composition in Parameter Space
Tenglong Liu, Jianxiong Li, Yinan Zheng, Haoyi Niu, Yixing Lan, Xin Xu, Xianyuan Zhan
2025-02-12
Summary
This paper talks about a new way to make AI agents learn and use skills more like humans do. The researchers created a system called PSEC that helps AI remember and combine different skills to solve new problems efficiently.
What's the problem?
Current AI systems aren't great at learning new skills without forgetting old ones, and they struggle to use what they already know to help them learn new things. This makes it hard for AI to adapt to new challenges the way humans can.
What's the solution?
The researchers developed PSEC, which works like a skill library for AI. It stores different skills as separate, easy-to-use modules that can be plugged in or combined as needed. PSEC also includes a smart system that figures out which skills to use based on the current situation. This allows the AI to mix and match its skills to tackle new challenges without having to learn everything from scratch.
Why it matters?
This matters because it could make AI much more flexible and efficient in solving real-world problems. By learning and combining skills more like humans do, AI could become better at adapting to new situations in fields like robotics, autonomous driving, or any area where machines need to make decisions in changing environments. It's a step towards creating AI that can continuously learn and improve itself, which could lead to more capable and useful AI systems in the future.
Abstract
Humans excel at reusing prior knowledge to address new challenges and developing skills while solving problems. This paradigm becomes increasingly popular in the development of autonomous agents, as it develops systems that can self-evolve in response to new challenges like human beings. However, previous methods suffer from limited training efficiency when expanding new skills and fail to fully leverage prior knowledge to facilitate new task learning. In this paper, we propose Parametric Skill Expansion and Composition (PSEC), a new framework designed to iteratively evolve the agents' capabilities and efficiently address new challenges by maintaining a manageable skill library. This library can progressively integrate skill primitives as plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient finetuning, facilitating efficient and flexible skill expansion. This structure also enables the direct skill compositions in parameter space by merging LoRA modules that encode different skills, leveraging shared information across skills to effectively program new skills. Based on this, we propose a context-aware module to dynamically activate different skills to collaboratively handle new tasks. Empowering diverse applications including multi-objective composition, dynamics shift, and continual policy shift, the results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC exhibits superior capacity to leverage prior knowledge to efficiently tackle new challenges, as well as expand its skill libraries to evolve the capabilities. Project website: https://ltlhuuu.github.io/PSEC/.