Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking
Chris Samarinas, Hamed Zamani
2025-04-08
Summary
This paper talks about a smart way to teach smaller AI models how to sort search results by learning from a bigger AI teacher and practicing like a student who explains their answers step-by-step.
What's the problem?
Current AI search tools either need expensive human help or giant AI models to understand complex searches, making them slow and hard to use for everyday needs.
What's the solution?
The method uses free web data and a big AI teacher to create training examples, then trains a compact AI to think through search rankings like solving puzzles, focusing on explaining why results match queries.
Why it matters?
This helps create faster, smaller AI search tools that work as well as giant models, useful for improving search engines and apps without needing massive computing power.
Abstract
We present a novel approach for training small language models for reasoning-intensive document ranking that combines knowledge distillation with reinforcement learning optimization. While existing methods often rely on expensive human annotations or large black-box language models, our methodology leverages web data and a teacher LLM to automatically generate high-quality training examples with relevance explanations. By framing document ranking as a reinforcement learning problem and incentivizing explicit reasoning capabilities, we train a compact 3B parameter language model that achieves state-of-the-art performance on the BRIGHT benchmark. Our model ranks third on the leaderboard while using substantially fewer parameters than other approaches, outperforming models that are over 20 times larger. Through extensive experiments, we demonstrate that generating explanations during inference, rather than directly predicting relevance scores, enables more effective reasoning with smaller language models. The self-supervised nature of our method offers a scalable and interpretable solution for modern information retrieval systems.