Leveraging Technology to Incentivize Student Effort in India
- Primary schools
- Students
- Student learning
- Digital and mobile
- Incentives
- Computer-assisted learning
- Online learning
Incentives to motivate students have traditionally focused on outcomes such as test scores, but this may not be effective if students do not understand the returns to effort or lack the self-control to exert effort now for rewards later. Using a technology-based math curriculum, researchers evaluated the impact of different incentive types—rewarding continuous effort in the classroom versus rewarding test performance—on student learning. Results indicate that effort-based incentives can be a more effective and cost-effective way of improving learning than incentives focused on test scores.
Policy issue
A key challenge to increasing learning levels is motivating students. Since student effort is costly to monitor, student incentives have traditionally been linked to outcomes such as test scores. However, rewarding continuous effort in the classroom could be more effective than rewarding final outcomes. Education technology provides a novel and potentially cost-effective means to monitor inputs such as time spent on a specific assignment or question, yet there remains little rigorous evidence on how best to integrate technology into classrooms instruction and learning processes. This evaluation examines whether continuous incentives for student effort can be a cost-effective way to help students engage better with technology and increase learning.
Context of the evaluation
The Motivation for Excellence Foundation’s (MFE) Nalanda project aims to cost-effectively integrate a tablet-based learning platform (“KA Lite”) into the local math curriculum. KA Lite uses offline Khan Academy instructional videos and interactive exercises, and allows teachers to track student effort and performance in real time. The platform is loaded onto low-cost tablets, which wirelessly connect to a local server.
The study took place in 4th-6th grade classrooms in Mumbai and Pune, India, including a mix of municipal, low-income private, and non-profit schools. Class sizes averaged 39 children, and the majority of classes were taught by Teach for India fellows in their first or second year of teaching. Classrooms also tended to have more boys than girls, with girls comprising only 40 percent of the sample.
Details of the intervention
Researchers partnered with MFE to evaluate the impact of building two types of incentives into the KA platform on student learning outcomes. Researchers randomly assigned each of the 45 classrooms to one of three groups:
- Input incentive: In 14 classrooms, the learning modules included incentives designed to encourage student effort. As students answered questions, they received instant feedback, allowing them to learn from their mistakes. If they could not answer the question correctly, students could watch a short video that explained the concept or see the full solution. Students could only finish the module when they answered 8 out of the last 10 attempted questions correctly, and each module was followed by a series of five questions. Students earned points for a combination of mastery and questions answered correctly, which they could use to buy rewards such as art supplies through an online store.
- Output incentive: In 14 classrooms, students worked through the same learning modules but did not need to reach a certain level of mastery in order to proceed. Instead, they could earn points to purchase rewards by answering test questions correctly at the end of a unit.
- Comparison group: In 17 classrooms, students worked through the same learning modules, but were not offered an incentive.
Each group completed two units of the curriculum. After each unit, students took non-incentivized tests as part of the standard school curriculum to measure learning gains. After the first unit, the input and output classrooms rotated to receive the other type of incentive, allowing researchers to evaluate whether the input incentive helped students learn how to earn higher rewards in the second unit.
Results and policy lessons
Rewarding continuous effort was more effective at improving test score performance than either the output incentives or no incentive. Students who received an input incentive scored 0.57 standard deviations better on the final tests than the comparison group, and 0.33 standard deviations than those who received output incentives. There was no significant difference between scores of students who received the output incentive and scores of students in the comparison group. The input incentive was nearly twice as cost-effective as the output incentive, at approximately $0.32 per 0.1 standard deviation increase in test scores.
Researchers suggest that these gains stem from addressing student’s tendencies to focus on the present and lack of information about the returns to exerting effort. In such classroom settings, frequent input-based rewards can be substantially more effective than output-based rewards at improving learning outcomes.