20 for 20: The tutor is calling - Targeted instruction by phone boosts learning across six countries

This is the first post of J-PAL’s 20 for 20: Partner Voices blog series, where we showcase stories of results and impact with our partners to celebrate our 20th anniversary. Read on to learn more about Youth Impact's work designing and evaluating interventions to help children stay in school and reach their full potential.

Over one billion children experienced some form of extended school closure from 2020 to 2022 (World Bank, 2022). Youth Impact, an NGO headquartered in Botswana, had for several years prior implemented Teaching at the Right Level (TaRL), an approach to teaching foundational literacy and numeracy, due to evidence of its effectiveness demonstrated by J-PAL affiliated researchers. During school closures, Youth Impact tested a phone-based adaptation of TaRL for mathematics that became ConnectEd, a one-on-one tutoring program that uses phones to connect high-quality instruction with students and parents. With ConnectEd, every week grade 3 to 5 students receive math problems delivered via text message along with a twenty-minute phone tutorial session. Lessons are tailored to the student’s previous knowledge and performance and cover basic operations (addition, subtraction, multiplication, and division).

ConnectEd was demonstrated to be effective through a randomized evaluation in Botswana that tested text messages alone compared to texts combined with individual phone calls (Angrist et al., 2022). These results prompted a coalition of NGOs, governments, multi-laterals, and researchers in five other countries (India, Kenya, Nepal, Philippines, and Uganda) to test and adapt phone tutoring programming in their context (Angrist et al. 2023). This coalition included J-PAL as a research partner, and the evaluation was funded by J-PAL’s Innovation in Government Initiative. These trials took place from 2020 to 2022, and represents some of the most rapid and comprehensive multi-country evidence generated on an education intervention. 

Key finding 1: Phone tutoring plus text messages demonstrated positive impacts in all six countries

On average, phone calls improved learning by 0.33 standard deviations, over a third larger than the median effectiveness of education interventions (Evans and Yuan, 2022). Although children who received phone tutoring learned more in every location, the greatest gains were in the Philippines and Uganda (0.45 and 0.89 standard deviations, respectively), where school closures were longer than in other countries. Results for text messages alone were more mixed, demonstrating statistically significant impacts only in the Philippines (0.09 standard deviations) and Uganda (0.21 standard deviations). See the working paper (Angrist et al., 2023) for in-depth results. 
 

Table 1.
The combination of targeted phone instruction plus text messages improved learning in every context in which it was tested

Key finding 2: Government and NGO tutors were equally effective at improving learning through phone tutoring

In addition to assessing the impacts of text messages alone compared to text messages plus calls, in two sites (Nepal and Philippines), we also randomly allocated students receiving ConnectEd to one of two groups: 1) tutored by government teachers or 2) tutored by individuals hired by an NGO. We found that impacts by type of tutor were remarkably similar—the share of students who had mastered the operation differed by at most 3 percentage points across NGO and government delivery. This provides further evidence that phone call tutorials can be delivered by a range of tutor profiles and supports its scalability.  

Key finding 3: Phone tutoring was highly cost-effective 

The primary tool required to implement and receive ConnectEd tutoring is a mobile phone, owned by nearly every household in countries where ConnectEd was tested (World Bank, 2021). Because ConnectEd builds on existing household infrastructure, the main costs were found to be related to content delivery and connecting with families. These included:

• training tutors 
• staff time spent preparing lesson content
• tutors’ time (for scheduling and making calls)
• mobile phone airtime 
• staff time for supervision and management. 

Costs per student for the duration of the program averaged $12. ConnectEd is a highly cost-effective program—just three hours of highly-targeted instruction improved numeracy.

Supporting systems: ConnectEd can be utilized as a complement to school and in emergency contexts 

We’ve shown that ConnectEd was highly effective in teaching students basic numeracy skills when schools were closed due to the Covid-19 pandemic. Even after schools have reopened, millions of children worldwide still face unexpected closures due to natural disasters, conflict, climate shocks, and diseases—as well as less acute circumstances like physical barriers or teacher absence. Building resilience into education systems by enabling quality provision of education during school disruptions is critical.

Families with lower incomes are often less equipped to support learning during prolonged closures, as many do not have access to resources that enable learning at home. As long as mobile phones are accessible to households and children are healthy and safe enough to learn, ConnectEd can maintain schooling continuity and accelerate learning. If used outside the classroom, ConnectEd can target students who need extra support.  

Results from our study also emphasize the substantial impacts of one-on-one tutoring and targeted instruction, demonstrated by other studies. Barriers to implementing more widespread individualized tutoring have been cost and access, which are substantially lower for phone tutoring, compared to in-person. The low-cost, accessibility, and ease of implementation can allow school systems to utilize more tailored approaches to address the persistent learning crisis.

Contact [email protected] or visit www.youth-impact.org for more information about ConnectEd.