Can LLMs Learn by Teaching? A Preliminary Study
Xuefei Ning, Zifu Wang, Shiyao Li, Zinan Lin, Peiran Yao, Tianyu Fu, Matthew B. Blaschko, Guohao Dai, Huazhong Yang, Yu Wang
2024-06-28
Summary
This paper talks about the concept of Learning by Teaching (LbT) in large language models (LLMs), exploring whether these models can improve themselves by teaching other models, similar to how human teachers learn by instructing students.
What's the problem?
In traditional machine learning, models usually rely on large amounts of human-generated data to improve their performance. This can be limiting and requires constant input from humans. The question arises: can LLMs learn and enhance their capabilities by teaching other models instead of just relying on external data? This potential for self-improvement could lead to more efficient training processes.
What's the solution?
To investigate this idea, the authors designed three methods that mimic how humans learn by teaching. These methods focus on: (1) observing feedback from students, (2) learning from that feedback, and (3) iteratively improving through teaching. The authors found that when strong models taught weaker ones, it led to better generalization—meaning the stronger model improved its own performance while helping the weaker model. They also discovered that teaching multiple students at once could be more effective than focusing on just one.
Why it matters?
This research is important because it opens up new possibilities for training LLMs more effectively and efficiently. By exploring the idea of LbT, we can potentially reduce the reliance on extensive human-generated data while enhancing the models' learning processes. This could lead to smarter AI systems that continuously improve over time, making them more adaptable and capable in various applications.
Abstract
Teaching to improve student models (e.g., knowledge distillation) is an extensively studied methodology in LLMs. However, for humans, teaching not only improves students but also improves teachers. We ask: Can LLMs also learn by teaching (LbT)? If yes, we can potentially unlock the possibility of continuously advancing the models without solely relying on human-produced data or stronger models. In this paper, we provide a preliminary exploration of this ambitious agenda. We show that LbT ideas can be incorporated into existing LLM training/prompting pipelines and provide noticeable improvements. Specifically, we design three methods, each mimicking one of the three levels of LbT in humans: observing students' feedback, learning from the feedback, and learning iteratively, with the goals of improving answer accuracy without training and improving models' inherent capability with fine-tuning. The findings are encouraging. For example, similar to LbT in human, we see that: (1) LbT can induce weak-to-strong generalization: strong models can improve themselves by teaching other weak models; (2) Diversity in students might help: teaching multiple students could be better than teaching one student or the teacher itself. We hope that this early promise can inspire future research on LbT and more broadly adopting the advanced techniques in education to improve LLMs. The code is available at https://github.com/imagination-research/lbt.