Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise
Rose E. Wang, Ana T. Ribeiro, Carly D. Robinson, Susanna Loeb, Dora Demszky
2024-10-07
Summary
This paper introduces Tutor CoPilot, a new AI tool designed to help tutors provide better support to students by offering real-time expert-like guidance during tutoring sessions.
What's the problem?
In education, especially for novice tutors, it can be challenging to provide effective instruction due to a lack of experience and resources. This is particularly an issue in underserved communities where students may not receive high-quality education. Traditional training for tutors can be expensive and time-consuming, creating barriers to improving educational quality.
What's the solution?
To address this problem, the authors developed Tutor CoPilot, which uses generative AI to analyze tutoring sessions and provide immediate, actionable suggestions to tutors. In a study involving 900 tutors and 1,800 K-12 students, they found that students whose tutors had access to Tutor CoPilot were more likely to master their subjects. The tool costs only $20 per tutor annually and helps tutors use effective teaching strategies, such as asking guiding questions instead of simply giving answers.
Why it matters?
This research is important because it demonstrates how AI can enhance the quality of education by supporting less experienced tutors. By making expert guidance accessible and affordable, Tutor CoPilot has the potential to improve learning outcomes for students in underserved communities, making high-quality education more equitable.
Abstract
Generative AI, particularly Language Models (LMs), has the potential to transform real-world domains with societal impact, particularly where access to experts is limited. For example, in education, training novice educators with expert guidance is important for effectiveness but expensive, creating significant barriers to improving education quality at scale. This challenge disproportionately harms students from under-served communities, who stand to gain the most from high-quality education. We introduce Tutor CoPilot, a novel Human-AI approach that leverages a model of expert thinking to provide expert-like guidance to tutors as they tutor. This study is the first randomized controlled trial of a Human-AI system in live tutoring, involving 900 tutors and 1,800 K-12 students from historically under-served communities. Following a preregistered analysis plan, we find that students working with tutors that have access to Tutor CoPilot are 4 percentage points (p.p.) more likely to master topics (p<0.01). Notably, students of lower-rated tutors experienced the greatest benefit, improving mastery by 9 p.p. We find that Tutor CoPilot costs only $20 per-tutor annually. We analyze 550,000+ messages using classifiers to identify pedagogical strategies, and find that tutors with access to Tutor CoPilot are more likely to use high-quality strategies to foster student understanding (e.g., asking guiding questions) and less likely to give away the answer to the student. Tutor interviews highlight how Tutor CoPilot's guidance helps tutors to respond to student needs, though they flag issues in Tutor CoPilot, such as generating suggestions that are not grade-level appropriate. Altogether, our study of Tutor CoPilot demonstrates how Human-AI systems can scale expertise in real-world domains, bridge gaps in skills and create a future where high-quality education is accessible to all students.