Does EdTech Substitute for Traditional Learning? The Impact of Computer-Assisted Learning in Russia
- Students
- Student learning
- Computer-assisted learning
The rapid expansion of education technology as a substitute for traditional learning, especially in the backdrop of the Covid-19 pandemic, raises the question of what level of education technology enables students to learn best. Researchers conducted a randomized evaluation to evaluate the impact of computer-assisted learning (CAL) on student test scores. The CAL program improved students’ math and language test scores, but increasing the time students spent on CAL did not lead to additional gains.
Problema de política pública
Educational technology (EdTech), such as computer-assisted learning, is an increasingly popular intervention to improve student learning. A 2017 study estimates that the global EdTech industry is projected to grow to more than US$340 billion by 2025. Moreover, the Covid-19 pandemic led to the large-scale replacement of traditional learning with EdTech due to school closures. Existing studies on the effectiveness of computer-assisted education found mixed results, with some studies identifying large, positive impacts on learning, but others finding no impacts. Can computer-assisted learning improve students’ learning in Russia, and how do results differ based on the intensity of programming?
Contexto de la evaluación
The Computer-Assisted Learning (CAL) software evaluated in the study is provided by one of the largest online technology companies in Russia. The provider’s platform has more than 10,000 items across various math and language sub-content areas for grades 2 to 4. The items and associated content areas align with national educational standards and curricula for primary schools.
Participants in the study included approximately 6,000 third-grade students in 343 primary schools in the Altai Krai and Novosibirsk provinces of Russia. Each third-grade class had an average of 18 students and one teacher that taught both math and language. Around 2 percent of the teachers were female.
Detalles de la intervención
Researchers evaluated the impact of a CAL program for third-grade students in Russia on math and language learning. The program utilized a graphics-based software with an attractive user-interface and dynamic, engaging tasks. The software allowed students multiple attempts per question and provided feedback after each response. Teachers could also track and compare student performance both overall and for specific content on the platform.
Researchers randomly assigned the participating schools into the following interventions:
- Low intensity group (115 schools): Third-grade students belonging to schools in this group received approximately 20-25 minutes per week each for math and Russian language CAL.
- High intensity group (113 schools ): Third-grade students belonging to schools in this group received approximately 40-50 minutes each for math and Russian language CAL.
- Comparison group (115 schools): Third-grade students belonging to schools in this group did not receive the CAL intervention.
Researchers surveyed a sample of third-grade students and all of their teachers and principals in each of these groups at the start of the Russian school year in October 2018 before the CAL program began. Students in the low and high intensity groups received the CAL program for six months from December 2018 until May 2019. Researchers conducted follow-up surveys at the end of the Russian school year in May 2019. In these surveys, researchers collected data on students’ math and Russian language end-of-year exam scores, students’ interest in studying math and language and time spent on subject-specific homework, and teachers’ time spent preparing to teach different subjects.
Resultados y lecciones de la política pública
The CAL program had positive effects on students’ math and language test scores, but increasing the program’s intensity, or time spent on CAL, did not lead to additional gains in test scores.
Math scores: For students in the low intensity group, the CAL program increased math test scores from 0.11-0.12 standard deviations. Students in the low intensity group who had below-median math scores prior to program implementation experienced the greatest gains in test scores. Students in the high intensity group also experienced similar positive effects on math scores, but the magnitude of the impact was roughly similar to the impact on the low intensity group. Thus, increasing the time spent on CAL resulted in no additional gains in math test scores.
Language scores: For students in the low intensity group, the CAL program increased Russian language test scores by 0.06-0.07 standard deviations. However, for the high intensity group, where the amount of time spent on CAL was doubled, the impact of the program was close to zero.
Other outcomes: Neither the low or high intensity program had effects on the time students spent on homework or the time teachers spent preparing for classes.
The authors suggest that the graphics and engaging nature of CAL may have led to its positive impacts. However, this positive effect might diminish at a higher intensity because of waning interest, increased fatigue, or distractions from using the computer for entertainment.
Taken together, these results suggest that CAL technology can improve academic achievement, but substituting traditional learning with CAL too much may be a misguided policy due to diminishing gains. Future research should continue to investigate the optimal level of CAL investments.