Peer Effects in Computer Assisted Learning: Evidence from a Randomized Experiment
- Students
- Student learning
- Digital and mobile
In China, students in districts with average incomes that are below the national average achieve lower levels of learning compared students in wealthier districts. Researchers introduced a computer-assisted learning program (CAL) to classrooms in Shaanxi Province to evaluate its impact on math test scores. The CAL program increased test scores on average and the cost of the program was halved by having two students share a computer. Furthermore, pairing students performing below the class average with students performing above the class average improved the effectiveness of CAL for both types of students.
Policy issue
A growing body of evidence shows computer-assisted learning (CAL) programs are an effective way to catch up students who are behind grade level on the material. However, the cost of a computer is an obstacle in many areas. One possible solution is to share computers between two students, thus lowering the cost per student. There has been little research on how sharing a computer between students could change the effectiveness of CAL programs: it is possible that the students would compete with each other for access to the computer, lowering the effectiveness of CAL. On the other hand, they could help each other understand the material, which would increase the effectiveness of CAL. Additionally, the impact of sharing a computer with a peer might depend on the characteristics of both students: pairing a low performingstudent with a high performing student might have a different impact than pairing a low preforming student with a similarly low performing student. Students of the same ability might be able to work together to learn the material more efficiently, while students of different ability levels might struggle to work together. How does sharing a computer while participating in a computer-assisted learning program affects learning outcomes for students?
Context of the evaluation
The evaluation took place in the Ankang prefecture, which is the poorest prefecture in the Shaanxi Province, a province which itself contains many of the poorest counties in China. Four out of the eight poorest counties in the Ankang prefecture were randomly selected to take part in the study. All 72 primary schools in these four counties took part in the evaluation, with half randomly selected to receive the CAL intervention. Within each primary school, the evaluation focused on grades three through six, as these grades were the target of the CAL intervention.
Details of the intervention
The evaluation took place during the 2011-2012 academic school year and included 7,881 third through sixth grade students. Half of the 72 schools were randomly selected to receive the CAL program and formed the treatment group, while the other half did not receive any CAL program and formed the comparison group. Within each classroom in a CAL school, students were randomly split into groups of two; these pairs received the CAL intervention together by sharing a computer. Pairs remained the same for the entire school year. Due to some classrooms having an odd number of students, six percent of the students were randomly chosen to receive the CAL intervention individually rather than sharing a computer with another student. This allowed the researchers to examine the effects of CAL programs as well as the effects of receiving CAL while sharing a computer with another student. Building on the evaluation performed by Mo et al., 2014, researchers also kept track of the ability levels of each student in a pair in order to learn how being paired with a weaker or stronger student changed the effects of receiving CAL programs. Researchers defined students as low-ability if they performed below the class mean, and high-ability if they performed above the class mean.
Students who received the CAL program had two 40-minute CAL sessions each week, which took place in school and were mandatory for all students in treatment schools. During CAL sessions, students watched instructional videos and then played mathematics games based on the material they were learning in their math classes. The videos were based on the uniform national curriculum (China’s most common curriculum), and were grade-specific. All students in the same grade across treatment schools received the same videos and exercises.
Results and policy lessons
Impacts of CAL on test scores: On average, the CAL sessions led to a 0.17 standard deviation increase in math test scores. This treatment effect is discussed in more detail in Mo et al., 2014. Researchers found that nine percent of the 0.17 standard deviation increase came from lower-performing students catching up to the rest of the class. Furthermore, lower-performing students who received CAL earned scores closer to higher-performing students than lower-performing students who did not receive CAL. This suggests that CAL could reduce performance gaps between low- and high-performing students within schools.
Impacts of sharing a computer during CAL sessions: The average effect of CAL was the same whether students received the program individually or in pairs. This indicates that there are positive peer effects, or positive interactions between the two students paired during the CAL exercises, which can compensate for the inconvenience of two students sharing a computer. This finding suggests that it may be possible to reduce the cost of CAL programming while maintaining its positive effects on test scores.
Impacts of the performance of the peer: The math scores of low-performing students (who performed below the class average at the beginning of the year) improved the most when paired with high-performing students (who performed above the class average at the beginning of the year). Additionally, high performing students’ math scores improved the most when paired with low-performing students. This suggests that some of the effectiveness of CAL can be driven by knowledge sharing between peers: high-performing students understand the material better after giving advice to low-performing peers, while low-performing students increase their understanding by learning from their high-performing peers.
Taken as a whole, these results suggest that CAL may be a cost-effective intervention to close gaps in student performance. CAL sessions increased student math test scores by an average of 0.17 standard deviations. Moreover, not only was the cost of CAL reduced by pairing students in CAL sessions, but pairing low-performing and high-performing students together increased the benefit of CAL sessions for both groups of students.