Exploring the promise of education technology

Around the world, there is widespread and growing interest in the use of technology in education. A recent industry projection estimated the global education technology—or “edtech”—industry to be valued at $252 billion by 2020. However, the rapid growth in use of edtech has outpaced rigorous research on its effectiveness. It’s important to step back and understand how different technologies affect student learning in order to ensure education technology serves to close gaps in educational equality, rather than widen them.

J-PAL North America Executive Director Quentin Palfrey and Karen Cator, CEO of Digital Promise, discuss how we can use edtech to fight poverty during this transformational time in education.

To that end, Maya Escueta (Columbia), Andre Joshua Nickow (Northwestern), Phil Oreopoulos (University of Toronto; Co-Chair, J-PAL's Education sector), and I recently released a review of more than 100 experimental studies (RCTs and RDDs) in education technology. The review paper examines evidence across several areas of education technology: access to technology, computer-assisted learning, technology-based behavioral interventions in education, and online learning. The paper provides useful guidance for education policymakers, school administrators, and companies seeking to adopt technology to improve student learning outcomes.

Quentin speaks with Karen about J-PAL North America’s strategy to assess what we already know works in edtech and address knowledge gaps.

Among the paper’s many findings, computer-assisted learning (CAL)—using educational software to help students develop particular skills—emerges as a particularly promising model in education, especially when used to support learning in math. The review suggests that this approach may be effective because software can adapt instruction to a student’s learning level, letting kids learn at the pace that works best for them. It can also provide teachers with immediate feedback on student performance, enabling them to more quickly identify students who are struggling and intervene to help. Studies showed CAL was most effective when used as an in-class tool or as mandatory homework support, essentially providing personalized tutoring on an individual level. 

Despite popular initiatives around the world that seek to provide computers to every student in a classroom, impact evaluations found that simply providing devices to students generally did not improve learning outcomes (although it had some positive impacts on computer use and proficiency). As the evidence on CAL shows, it may matter what type of software is available on the device.
 
Another common edtech approach is use of technology-based behavioral interventions, such as text message nudges, to encourage students and parents to take certain actions. The review paper finds that these interventions consistently improved learning outcomes throughout different stages of the education “lifecycle,” from early childhood to post-secondary.

These efforts were most effective when focused on encouraging specific, one-time actions, such as completing course registrations, filing financial aid paperwork, and suggesting questions parents should ask their children to encourage learning. Though effect sizes were somewhat modest relative to CAL, the review found that technology-based nudges in education can be highly cost-effective given the generally low cost of implementation.
 
Though online courses have exploded in popularity over the past decade, research on effectiveness is still nascent. The review found that students in classes in which instruction was conducted in person generally outperformed their peers in online-only classes. But students in “blended” courses, which have both online and in-person components, typically performed just as well as students in in-person only courses, suggesting that blended courses could help drive down teaching costs.

In light of the buzz and sizeable investment in edtech, this review paper aims to take stock of what we currently know from the experimental evidence about what works. This review hopes to advance the knowledge base by identifying and discussing the most promising uses of edtech to date and highlighting areas that merit further exploration.

To learn more about these findings, read the review paper and visit our Education page to see more results from randomized evaluations.