Mobile Monitoring to Improve Adult Learning in Niger
- Students
- Teachers
- Rural population
- Student learning
- Digital and mobile
- Tracking
Adult education programs are an important part of the education system in many developing countries. Yet identifying the impact of such programs on learning outcomes can be difficult, in part due to the challenge of monitoring teachers’ attendance and effort. Researchers tested whether making weekly calls to teachers, students, and the village chief during an adult education program in Niger could increase teacher accountability and improve student learning. Both the education program and the weekly calls increased students’ scores in reading and math, and researchers suggest that this was due to a variety of factors, one being an increase in both teacher and student motivation.
Policy issue
Adult education programs are an important part of the education system in many countries. Yet identifying the impact of such programs on learning outcomes can be difficult, in part due to the challenge of monitoring teachers’ attendance and effort. In rural areas of many low-income countries, teacher absence is a widespread problem. Mobile phone technology could enable governments and NGOs to communicate with remote villages on a regular basis, potentially reducing the cost of monitoring teachers and other service providers. Similarly, mobile communication could enable community members and learners to provide more timely feedback on teacher performance, thereby increasing community engagement in the education system. Can introducing a simple mobile phone-based monitoring component into an adult education program increase teacher effort and improve students’ learning?
Context of the evaluation
Teacher absenteeism is a substantial problem across West Africa, with 27 to 40 percent of teachers in primary schools estimated as absent. Monitoring teachers’ attendance and effort, which has improved teacher absenteeism and student learning inother contexts, is expensive and logistically challenging due to limited infrastructure and weak institutions.
In Niger, literacy levels are particularly low: 30 percent of adults were classified as literate in 2012, the last year in which data was available, and it is estimated to be lower still among women and in rural areas. In the two regions where this study was implemented, Maradi and Zinder, it is estimated that 10 percent of women had attended any school.
Although members of study participants’ households had limited education—two years of schooling on average—mobile phone use was common; 63 percent of households owned a mobile phone.
Details of the intervention
In 2014, researchers partnered with Catholic Relief Services (CRS) and the Government of Niger’s Ministry of Non-Formal Education (MNFE) to test whether adding weekly check-in calls to students, teachers, and village chiefs during an adult education program could increase teacher effort and improve student learning outcomes. 131 villages in two rural regions were randomly selected to participate in the study. Villages were then randomly assigned into one of three groups:
- Adult education only: In 54 villages, the local implementing partners implemented a two-year adult education program (basic literacy and numeracy) with normal monitoring visits by CRS and MNFE staff.
- Adult education + weekly mobile phone calls: In 57 villages, in addition to the adult education program, teachers, village chiefs, and two randomly selected students received weekly phone calls over a six-week period. A variation of this approach, in which only teachers were called, was implemented in a subset of these villages during the second year of the program. During the calls recipients were asked a series of questions about the number of classes held, the number of hours for each class, and the number of students who attended.
- Comparison group: 20 villages continued in the status quo, with no program.
Teachers were either from the community or a neighboring community, and were selected and trained by MNFE. Classes were separated by gender and held five days per week for three hours a day.
Researchers collected data on participants’ reading and math abilities to measure impacts on learning prior to the start of the program. They also collected administrative and survey data on teachers, including teachers’ motivation and perceptions of the monitoring program.
Results and policy lessons
Both the adult education program alone and adding weekly calls improved students’ learning.
Student learning: The adult education program improved students’ reading test scores by 0.22 standard deviations, meaning that by the end of the program, students in the standard adult education classes could successfully complete eight additional reading tasks (e.g. reading letters, syllables, and words) as compared to those in the comparison villages. Effects on math scores were similar; the program increased adults’ math scores by 0.19 standard deviations relative to the comparison group, meaning that students were able to correctly complete 2.7 additional math problems by the end of the program.
Weekly calls: Adding weekly calls improved students’ reading scores by an additional 0.15 standard deviations, meaning that by the end of the program, students in the mobile villages were able to complete an additional six reading tasks as compared to those in the standard education program. Weekly calls further increased math scores by .12 standard deviations: students were able to complete an additional 1.75 math tasks as compared to those in the standard adult education villages by the end of the program.
The overall results suggest that the adult education program was successful in improving students’ learning outcomes over time for math, but not for reading. While the weekly calls led to improvements in reading test scores in the first year, there were no additional benefits observed in the second year. However, students in villages with weekly calls improved their math skills over the two-year period. The individual students who received the weekly phone calls saw even more improvement; their test scores were four times higher than those who were invited to the standard adult education program. Similarly, the weekly calls did not change teachers’ attendance, but it may have affected teachers’ classroom preparation and effort, as measured by their ability to pass MNFE’s performance exam.
Mechanisms: The researchers suggest that some results were driven by increased teacher motivation and lower student dropout. Because data on teacher attendance was limited, disentangling the mechanisms was difficult. Given the cost-effectiveness of the intervention, however, researchers concluded that adding phone reminder calls to programs could be a low-cost way to improve adult learning outcomes.
Based on the results from this evaluation, the Ministry of Primary Education agreed to pilot and test this methodology in its primary schools.