DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
Jongwoo Ko, Tianyi Chen, Sungnyun Kim, Tianyu Ding, Luming Liang, Ilya Zharkov, Se-Young Yun
2025-03-11
Summary
This paper talks about DistiLLM-2, a smarter way to shrink big AI models into smaller ones by teaching them to copy the best answers and avoid mistakes, like a student learning from both right and wrong examples.
What's the problem?
When copying big AI models into smaller ones, most methods treat all answers the same way, even bad ones, which makes the smaller models less accurate and efficient.
What's the solution?
DistiLLM-2 uses a contrastive method that rewards the smaller model for copying good answers from the big model and penalizes it for repeating its own mistakes, making it learn faster and better.
Why it matters?
This helps create smaller, faster AI models that work almost as well as huge ones, making advanced AI cheaper and easier to run on phones or laptops.
Abstract
Despite the success of distillation in large language models (LLMs), most prior work applies identical loss functions to both teacher- and student-generated data. These strategies overlook the synergy between loss formulations and data types, leading to a suboptimal performance boost in student models. To address this, we propose DistiLLM-2, a contrastive approach that simultaneously increases the likelihood of teacher responses and decreases that of student responses by harnessing this synergy. Our extensive experiments show that DistiLLM-2 not only builds high-performing student models across a wide range of tasks, including instruction-following and code generation, but also supports diverse applications, such as preference alignment and vision-language extensions. These findings highlight the potential of a contrastive approach to enhance the efficacy of LLM distillation by effectively aligning teacher and student models across varied data types.