RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging
Bowen Wang, Haiyuan Wan, Liwen Shi, Chen Yang, Peng He, Yue Ma, Haochen Han, Wenhao Li, Tiao Tan, Yongjian Li, Fangming Liu, Yifan Gong, Sheng Zhang
2025-10-27
Summary
This paper introduces a new method, called RECALL, for continually updating large language models (LLMs) – like the ones powering chatbots – without making them forget what they already learned.
What's the problem?
Large language models are amazing, but they struggle with 'catastrophic forgetting'. This means when you teach them something new, they tend to lose their ability to do older tasks. Existing solutions often require access to old data, which isn't always available, or they force you to sacrifice performance on either new or old tasks. The core issue is how to integrate new knowledge without disrupting the existing knowledge base within the model.
What's the solution?
RECALL tackles this by focusing on the internal 'thought processes' of the LLM, specifically the hidden representations within its layers. It compares how different versions of the model (before and after learning something new) represent information, looking at common patterns. Then, it intelligently merges the models' parameters, carefully adjusting them to preserve general knowledge in the early layers and allow for task-specific learning in the later layers. This is done without needing any of the original training data.
Why it matters?
This research is important because it provides a way to continually improve LLMs without the drawbacks of previous methods. It allows for seamless integration of new information and prevents the model from forgetting what it already knows, all without needing to store and re-process old data. This makes it much more practical to keep these powerful models up-to-date and adaptable in real-world applications.
Abstract
We unveil that internal representations in large language models (LLMs) serve as reliable proxies of learned knowledge, and propose RECALL, a novel representation-aware model merging framework for continual learning without access to historical data. RECALL computes inter-model similarity from layer-wise hidden representations over clustered typical samples, and performs adaptive, hierarchical parameter fusion to align knowledge across models. This design enables the preservation of domain-general features in shallow layers while allowing task-specific adaptation in deeper layers. Unlike prior methods that require task labels or incur performance trade-offs, RECALL achieves seamless multi-domain integration and strong resistance to catastrophic forgetting. Extensive experiments across five NLP tasks and multiple continual learning scenarios show that RECALL outperforms baselines in both knowledge retention and generalization, providing a scalable and data-free solution for evolving LLMs.