Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic
Mohammad Zbeeb, Hasan Abed Al Kader Hammoud, Bernard Ghanem
2025-09-03
Summary
This paper explores a way to transfer reasoning skills learned by large language models to other models without needing to retrain them extensively, essentially 'copying' the ability to think through problems.
What's the problem?
Training large language models to perform complex reasoning tasks, like solving math problems or writing code, usually requires a lot of computational power and time using methods like reinforcement learning. This is expensive and inefficient, and it would be great if we could reuse reasoning abilities once they've been developed.
What's the solution?
Researchers took two identical language models and trained one using standard methods (supervised fine-tuning) and the other using a more advanced technique (group relative policy optimization) designed to improve reasoning. They then calculated the difference between the two models' internal settings – this difference is called the 'reasoning vector'. Adding this vector to other models actually improved their reasoning performance on various tests, while subtracting it made them worse, proving the vector really does contain reasoning ability. It's like isolating the part of a brain that's good at problem-solving and transferring it to another brain.
Why it matters?
This work is important because it offers a practical way to boost the reasoning capabilities of existing language models without the huge cost of retraining. It allows us to recycle the computational effort already invested in developing reasoning skills, making more powerful AI accessible and efficient. It's a step towards making AI reasoning more reusable and less resource-intensive.
Abstract
Large language models often require costly optimization, such as reinforcement learning, to master complex reasoning tasks. This work demonstrates that reasoning ability, once learned, can be extracted and transferred between models as a compact task vector. We source two publicly available, identically initialized Qwen2.5 models, one fine-tuned with supervised fine-tuning (SFT) and the other with group relative policy optimization (GRPO) on the same dataset. From these, we extract a reasoning vector: v_{reason} = theta_{GRPO} - theta_{SFT}. We hypothesize that this vector captures the reasoning capability instilled by reinforcement learning while factoring out shared knowledge from the SFT process. When added to compatible instruction-tuned models through simple arithmetic, this vector consistently improves performance across diverse reasoning benchmarks: GSM8K (+4.9%), HumanEval (+4.3%), SciQ (+1.7%), and BigBenchHard (+12.3% for the 1.5B model). The performance improvements persist under adversarial conditions. Conversely, subtracting the vector causes significant performance degradation (-11.8% on GSM8K), demonstrating the vector's strong contribution to the model's reasoning abilities. This work shows how reasoning capabilities, typically developed through expensive training, can be extracted from existing open-source models and reused through simple tensor arithmetic, offering a practical way to enhance models by recycling prior computational investments.