Impacts and Challenges of a Teacher Training Program in Nepal
Training teachers to improve teacher quality and student learning is a common strategy taken by policymakers. In Nepal, researchers conducted a randomized evaluation of a government teacher training program for math and science teachers to evaluate the impact on student learning in secondary schools. The training program had little or no impact on student learning. Through additional qualitative and quantitative work, researchers identified challenges in the training program’s design, including poor management of training sessions, lack of time and resources to implement new teaching practices, insufficient post-training accountability and support, and a mismatch in the training content difficulty and teachers and students’ ability levels.
Problema de política pública
In low- and middle-income countries, the quality of public services, like education, healthcare, and infrastructure, is often low. Training teachers is a common strategy among policymakers to develop teachers’ pedagogical skills and subject knowledge, with the ultimate goal of improving student learning. However, governments may face challenges in designing and implementing teacher training programs at scale that adequately match the varied skill levels of teachers and the learning needs of students they serve, and that account for the resource constraints within schools to participate in trainings or incorporate new practices. Evaluations of teacher training programs in low- and middle-income countries show mixed results, which is further complicated by the variation in the kinds of training programs being evaluated.
With these complications in mind, what are the impacts of an at-scale government teacher training program on student learning?
Contexto de la evaluación
In Nepal, student learning in government schools is low, particularly for secondary school students: among 9th grade students, more than 20 percent cannot divide six by three and nearly 40 percent cannot calculate the area of a square. Secondary school enrollment rates have been increasing sharply, with enrollment rising from 35 percent in 2008 to 66 percent in 2017, near the start of the evaluation, compounding the challenge of how to improve student learning. Among 9th and 10th grade students in Nepal, 97 percent attend government schools.
Secondary school teachers in public schools are required to have three to four years of pre-service training conducted by universities. Trainings conducted during a teacher’s career take place at government regional Educational Training Centers (ETCs). The national School Sector Reform Program (SSRP) ran from 2009-2015 and, in principle, required teachers to attend three ten-day training sessions over five years. However, by 2017, only 31 percent of math teachers and 24 percent of science teachers had received any SSRP training in their subject areas. Evaluations also suggested that the trainings were “too theoretical” and had little impact on teaching practices.
Subsequently, the Government of Nepal rolled out a new set of teacher trainings as part of the School Sector Development Program (SSDP)—a plan to improve school quality between 2016 and 2023. Under the SSDP trainings, participants attended ten days of in-person training at ETCs, followed by five days of self-study work focused on developing ten lesson plans and completing subject-specific learning activities that could be incorporated into their teaching plans. Most trainings took place during school days, and teachers received per diems for their days at the ETC. Teachers’ grades for their training participation and project work were to be included in their annual performance reviews. Unlike the SSRP, the government designed the SSDP trainings centrally, focusing on challenging math or science concepts from the 9th and 10th grade curricula and demonstration-based teaching methods with teaching aids made from local materials.
Detalles de la intervención
Researchers partnered with the Government of Nepal to evaluate the impact on student learning of the SSDP trainings for secondary math and science teachers in 203 government schools.
Researchers randomly selected sixteen districts from across all seven provinces in Nepal to participate in the study. Researchers then sorted schools within each district into two categories: “priority” schools had no teachers who had completed the prior SSRP training, while “non-priority” schools had at least one SSRP-trained teacher. Within each district, researchers randomly selected about eight schools from the priority group and four from the non-priority group, and assigned half of the selected schools in both groups to receive access to the SSDP teacher training. This resulted in the following groups:
- Training group: All 9th and 10th grade math and science teachers working at schools assigned to the training group received an invitation to participate in the SSDP teacher training program in late 2017. Most SSDP trainings then ran early in the 2018-2019 school year.
- Comparison group: Teachers in the comparison group did not receive an invitation to or permission to attend the SSDP training until May 2019, after the evaluation was complete.
To measure the impact of the training invitations on student learning, researchers administered two hour-long, in-person math and science assessments to nearly 12,000 9th and 10th grade students across the 203 schools at the end of the 2018-2019 school year. The assessments measured overall math or science achievement and achievement on math or science items most closely tied to content from the SSDP training. Researchers also indirectly tested teacher subject knowledge by asking teachers to provide feedback on what they thought was the correct answer to each question, how well each question aligned with Nepal’s curriculum, and more.
The researchers also collected a range of mostly qualitative data, including monitoring data on the training sessions, head teacher and student reports, a phone survey with a subset of trainers and training participants, and a small number of in-depth, in-person interviews with teachers, trainers, and other school actors.
Resultados y lecciones de la política pública
Inviting teachers to participate in the SSDP trainings led to little improvement in student learning, and may have reduced learning among the strongest students. Through additional mixed-methods work, researchers identified several potential explanations for this, including low rates of participation by invited teachers, poor management of training sessions, limited time and weak motivation for implementing new teaching practices, and a mismatch between the difficulty of the training content and teachers’ and students’ ability levels.
Student test scores: Researchers found no evidence that inviting teachers to participate in the SSDP training program improved student learning. On average, 10th grade students attending schools in the training group received the same math and science assessment scores as students in the comparison group schools, including on assessment questions closely aligned with the SSDP training content. Similarly, 9th graders in the training group received the same or slightly worse assessment scores as students in the comparison group.
Teacher participation rates: In the training group, only 61 percent of math teachers and 42 percent of science teachers at the endline survey participated in the SSDP trainings, limiting the potential impact of the program. Many teachers simply did not accept the invitation and attend the training, with only 69 percent of math teachers and 51 percent of science teachers attending the trainings. Reasons for this included that they did not expect to learn much, were preparing for teacher exams, inadequate per diems or accommodations, and/or personal reasons. However, some schools also declined to send their teachers to the training as finding substitutes was difficult.
Implementation of SSDP training sessions: Administrative efforts to make sure that trainers had the adequate skills and resources, like information, time, facilities, and materials, were lacking. In addition, trainers stated they had little time or guidance to prepare training content and some ETCs did not have trainers with sufficient math and science expertise.
Teacher subject knowledge: The SSDP training did not change teachers’ subject knowledge. Some teachers had weak knowledge of core math or science concepts prior to the training, which may have impeded some teachers from benefitting from the program, particularly training content related to more advanced math and science concepts.
Student preparation levels for secondary school: Many students entered 9th and 10th grade with below-grade-level knowledge, which may have been misaligned with the advanced concepts the SSDP training taught teachers to focus on. However, even higher-performing students did not benefit from their teachers’ attending the SSDP training.
Teaching practices: Teachers reported insufficient resources and time to prepare and implement lessons, which likely impacted their ability to implement new teaching practices. There was also a lack of accountability for implementing new teaching practices within schools, with teachers reporting little enforcement from school administration. Head teachers and trainers also reported inadequate motivation from teachers.
Program costs: SSDP trainings cost about USD$130 per teacher, or USD$2.60 to $3.00 per student. This cost is similar to that of training interventions from other contexts that have been found to be effective in improving student learning. This suggests that the trainings can be improved to be effective, or they could be replaced with interventions that are more cost-effective.
This evaluation suggests three broad policy lessons. First, it reveals the challenges governments and teachers may face when designing, implementing, or participating in teacher trainings. Second, improving the governance of public service activities requires not only strengthening accountability, but also ensuring that public servants have adequate time, capacity, and other resources to do their work well. Third, improving public service quality may require not only improvements in governance, but also improved tailoring of policy design to context.
While the implementing partner did not directly use the evaluation results, these findings have shaped governmental decisions aimed at enhancing educational quality. The study’s impact is evidenced by the government’s subsequent willingness to authorize another randomized controlled trial, this one focused on phone-based tutoring, when prompted by one of the researchers. Furthermore, the study provided a rigorous assessment of secondary students’ learning levels, corroborating the broadly acknowledged concern about their deficiencies. Both the Ministry of Education and various development partners have since underscored the issue of low learning levels, drawing from the insights provided by this research.
Schaffner, Julie, Paul Glewwe, and Uttam Sharma. “Why Programs Fail: Lessons for Improving Public Service Quality from a Mixed-Methods Evaluation of an Unsuccessful Teacher Training Program in Nepal.” Working paper. February 2023.