Moving from evidence to policy change

Today we launch an expanded section of our website called Evidence to Policy, which explores how J-PAL affiliated researchers and staff, along with government and NGO partners, have drawn on evidence produced by randomized evaluations to inform policy changes around the world.

The new section includes 17 case studies from J-PAL’s broad body of experience learning how evidence can be applied. We think these examples are particularly illustrative of the many ways randomized evaluations are contributing to policymaking. While sometimes studies directly help inform decisions to scale up (and often adapt) programs that have been proven to be effective, they also influence the policy process by providing general behavioral insights that can be used to design new programs. In other cases, studies lead to decisions to scale back interventions shown to be ineffective.

To underscore the diversity of this influence, we have outlined six pathways that we believe help capture the many ways that evidence can shape policy and practice:

• Shifting global thinking: Knowledge generated by randomized evaluations has fundamentally shaped our understanding of social policies.
  
  
• Institutionalizing evidence use: Organizations, including governments and large NGOs, have institutionalized processes for rigorously evaluating innovations and incorporating evidence into their decision-making.
  
  
• Applying research insights: Lessons from randomized evaluations have informed the design of programs.
  
  
• Adapting and scaling a program: Programs originally evaluated in one context have been adapted and scaled in others.
  
  
• Scaling up an evaluated pilot: Partners have replicated and expanded a successful evaluated pilot to similar contexts.
  
  
• Scaling back an evaluated program: Partners have scaled down, redesigned, or decided to not move forward with programs that were evaluated and found to be ineffective.

Together, more than 400 million people have been directly reached by programs that were scaled up after being evaluated by J-PAL affiliated researchers and found to be effective. We arrive at this figure by counting beneficiaries and program participants who were reached after the completion of a randomized evaluation that led to a decision to scale up. These include:

• The millions of children in Ethiopia, India, Kenya, Nigeria, and Vietnam reached by Evidence Action’s school-based deworming programs after evaluations in Kenya found that providing deworming tablets in schools improved health and increased school attendance.
  
  
• The millions of Indonesian families who benefit from tangible proof of eligibility for social assistance programs after the government partnered with researchers to evaluate a pilot scheme that provided identification cards to eligible families with information on their subsidized rice entitlements.
  
  
• The tens of thousands of poor households in Bangladesh and India who have exited extreme poverty after the scaling up of the “Graduation approach,” a multifaceted livelihood program first designed by Bangladeshi NGO BRAC that has been shown to help the poorest women and their households transition into more sustainable livelihoods.
  
  
• The millions of children in India and a small but growing number of African countries who are learning through the “Teaching at the Right Level” approach, piloted by Indian NGO Pratham and evaluated and refined over fifteen years. An iterative process of research, design, and evaluation produced a pedagogical method proven to improve learning by reorienting teaching to the learning level of the student.

Millions more have been impacted by the many other ways in which evidence from randomized evaluations has informed policy, but beneficiaries are more difficult to count. These include:

• A shift in thinking by donor and development agencies to eliminate user fees after more than a dozen evaluations showed that charging fees for preventive health products, such as malaria bednets, dramatically reduced take-up.
  
  
• A transformation in public service delivery in India following an evaluation of a fund flow reform scheme that reduced corruption and improved program efficiency by linking the distribution of central government funds to actual expenditures in India’s flagship work-for-pay program. The evaluation was cited by the central government in asking all states to adopt the reform.
  
  
• The institutionalization of rigorous evaluation into government policymaking and program design processes in Peru’s Ministry of Education. With support from J-PAL and Innovations for Poverty Action, the government opened MineduLAB—a dedicated unit within the ministry that designs, evaluates, and scales effective solutions to educational challenges.
  
  
• The decision to scale back a biometric health provider monitoring program in Karnataka state, India, after an evaluation showed that the government was unwilling to enforce penalties associated with provider absenteeism. Ending the use of the technology saved taxpayers’ money and avoided an ineffective drain on government resources.

To capture our thinking on moving from evidence to policy change, we have also introduced an Evidence to Policy resources page. Here you will find reflections by J-PAL affiliates and staff on how to leverage evidence to inform policy, including practical frameworks and guidance for adapting evidence from one context to another, assessing whether a program is ready for scale, understanding the value of evaluation in scaling policies, and more.

The Evidence to Policy page will continue to expand over time as we develop new case studies, blog posts, and videos that explore how researchers, policymakers, program implementers, and others have leveraged the power of randomized evaluations to drive policy change.

We would love to hear your feedback and welcome ideas for new case studies. Please email us at [email protected].