What have we learned about building a culture of data and evidence use in government?

This is the second of two posts about GPI. Read the first post in this series.

Earlier this month, we wrote about how partnerships funded by J-PAL’s Government Partnership Initiative (GPI) have helped increase governments’ use of data and evidence in policymaking in places like Chile, India, Peru, and Zambia.

One common theme across all of GPI’s partnerships is that they are led by a champion in government who is making evidence-informed policy a priority. We could not contribute data and evidence to policy decisions without their leadership and dedication.

But in addition to great leadership, what have we learned about how governments are building a culture of using data and evidence in policymaking? And about how J-PAL and other evidence-to-practice organizations can best support them in these efforts?

To start to answer these questions, we conducted in-depth interviews with stakeholders from eleven of our government partnerships in Brazil, Chile, Colombia, and Peru. We’ll share the full findings later this year, but in the meantime, some early insights about what’s working have emerged.

1. Make it someone’s job to use evidence.

Applying evidence to design better policies and programs takes time and resources. In GPI, we've seen that when an official or department is committed to prioritizing evidence-informed policy, allocating even a small amount of resources and personnel time to using evidence is an important way to start to build a culture to support data-driven and evidence-informed decisions.

Some governments can and do hire staff to do this on their own, but others don’t have the resources. In these cases, temporarily placing policy and research staff to work within a government office or closely with them to achieve a particular goal can be effective (for example, see the India case from our last post). Working in a government office can be an effective way to build trust, better understand their priorities, and spot policy windows where evidence could be useful.

Embedded staff can also be useful to government officials, who may not have the time to compile the data and evidence they need to make policy decisions, by producing summaries of existing evidence or doing quick analyses of administrative data.

This model is not new; organizations like the Overseas Development Institute and others have embedded technical staff in governments for years. We’re just adding our voice to the chorus of organizations saying that it can work, even (and maybe especially) as a way to drive the use of evidence in policy.

In hiring these staff, we’ve found that the most important qualities are previous government experience in the country or region, knowledge of existing evidence and research methods, data analysis skills, and an enthusiasm for supporting their government colleagues and being responsive to their needs. Some governments have gone on to hire these temporary staff as full-time employees, including one of our partners in Brazil.

2. Help governments make better use of the data they already collect.

Many governments collect vast amounts of data, but need support to organize it in a usable format and identify use cases to improve program management. Creating user-friendly charts, maps, tables, and dashboards with key stats on program implementation, or combining previously disconnected datasets, can help officials see how their data can be useful in driving better program management decisions.

Administrative data can also be used to run low-cost evaluations that can demonstrate the usefulness of testing and refining programs. Several of our government partners highlighted the importance of improving their administrative data systems to accelerate their transition to evidence-informed policymaking.

In Brazil, GPI funded the salary for a research associate to sit inside a state security department to help organize and analyze its data more effectively. She helped them develop a platform that allows police to view real-time data on crime, police presence, and environmental factors, to inform decisions about dispatching officers.

Research associate in Brazil analyzes data for a state security department

3. Invest in long-term partnerships.

Having a long-term presence in a country or a long-term relationship with a government fosters greater trust and mutual understanding. Long-term partnerships between governments and researchers can help civil servants understand the different ways that evidence can be used and help researchers make their research more useful for policy.

By building a reputation and knowledge among many people in a government, long-term partnerships allow us to identify and respond quickly to policy windows where evidence could be useful, know the right people to work with, and maintain partnerships despite transfers of key officials or administration changes after elections.

Working together on multiple projects, and not just impact evaluations, is also more conducive to evidence use and institutional learning than a one-time evaluation.

In Peru, J-PAL and Innovations for Poverty Action used a GPI grant to begin collaborating with the Ministry of Women and Vulnerable Populations in 2016. Our first project was to create an evidence repository of all relevant completed impact evaluations related to the Ministry’s priority questions on reducing domestic violence, which the Ministry used to inform its strategy on reducing domestic violence.

As a result of this work, the Ministry developed a new randomized evaluation with J-PAL affiliated researcher Erica Field to evaluate the impact of one of the Ministry’s flagship domestic violence prevention programs.

4. Take on quick wins that can build trust and demand for evidence.

Starting a relationship with simple, low-cost evaluations, descriptive data analyses, or trainings tailored to government priorities can open the door for future collaboration by demonstrating the usefulness of data, evaluation, and existing evidence.

Collaborating with government schools that train civil servants can also help make training resources available to a much larger number of agencies.

Recently GPI funded a partnership with Brazil’s National School of Public Administration (ENAP) to develop an online course on evaluating social programs in Portuguese. The course, which is freely available on ENAP’s online learning platform for all Brazilian citizens, will make our capacity-building materials more widely available. Given that more than 1,000 Brazilian civil servants recently completed the course, we are enthusiastic about this model for engaging a wide range of officials across agencies.

5. Dedicating a small sum of money to evidence use can ease large constraints.

When directed to the right people at the right time, even small amounts of money can reduce seemingly large barriers to using data and evidence in decisions.

Often, small sums can allow either governments or organizations like J-PAL to hire someone whose job it is to use evidence, collect relevant data, or invest in the software they need to process information. Providing quick-turnaround grants after an evaluation is complete can also support evidence-informed scale-ups or reforms, or help set up a government lab that incentivizes greater use of evidence. Governments may not be able to fund these activities due to budget constraints, lengthy contracting requirements, or conflicting priorities, and a small grant can have a big impact.

For example, in India, just US$70,000 of GPI support enabled J-PAL South Asia to support a government partner’s efforts to lead the national scale-up of a funds-flow reform to a major government program.

Stay tuned

We'll be sharing more insights from GPI throughout 2018. Based on what we’ve learned so far, we hope to expand our investments in two successful models going forward:

1. Embedding J-PAL staff within governments to support the use of evidence in policy design in key sectors like education, health, and social protection.

2. Working with governments to make the administrative data they already collect more useful for day-to-day decision-making and evaluation.

We're always looking for new partners in these efforts. If you're interested in learning more about J-PAL's Government Partnership Initiative, email us at [email protected].