AI-Powered Tutoring: Unleashing the Full Potential of Personalized Learning with Khanmigo

Tutoring stands out as a highly effective educational policy for improving student outcomes, but its implementation is hindered by issues of scalability and cost. One solution involves equipping teachers with enhanced skills in utilizing Computer Assisted Learning (CAL) to simulate the tutoring experience at a lower expense. A recent evaluation of this approach, called "Khoaching with Khan Academy" demonstrated meaningful improvements in standardized math test scores for elementary students whose teachers were randomly assigned to receive such assistance, as compared to students whose teachers did not benefit from it. Although this intervention successfully enhanced average math scores, the study also highlighted significant variations in CAL practice time both within and across classrooms receiving the treatment. To improve engagement of the Khan Academy platform and increase practice time and progress, the KWiK program will leverage Khanmigo, an AI-powered virtual assistant developed by Khan Academy, as a personalized tutor to support student’s progress with Khan Academy assignments. This project will compare student engagement and progress between classrooms with Khan Academy’s current one-time training for Khanmigo usage versus those with teachers assigned to a proactive ‘khoach’ who meets regularly with teachers to help guide and assist high dosage practice.  Within these classrooms, the project will also randomly test different ways to encourage Khanmigo usage. Currently, students that have access to Khanmigo must actively click on an icon at the corner of their page to open a window to work with it.  This setup will be compared to having Khanmigo pro-actively pop-up after students answer incorrectly, or at the start of a Khan Academy practice session. We will also test what Khanmigo says to students to identify better ways to increase Khanmigo engagement. The study will yield valuable quantitative and qualitative evidence regarding the potential of leveraging powerful artificial intelligence technology for tutoring purposes. This will be the first study to test the effects of ChatGPT-based AI tutoring to help improve learning performance, and compare these effects with added person-based tutoring.