J-PAL at 20: Supporting the use of rigorous evidence in German development cooperation

How can we support our policy partners to institutionalise the use of rigorous evidence? This blog post, the second in a series exploring J-PAL's research and policy activities across Europe, looks at our partnership with German development cooperation actors to facilitate the use of evidence in project planning and strategic decision making. Relevant for our work with development actors as well as government partners across Europe, it highlights the importance for organisations like J-PAL to fulfil a role as an “evidence broker,” bridging the gap between research and policy. 

The increase in the production of rigorous impact evaluations in the past twenty years has been met by a growing interest among development actors in how to draw on this evidence base when designing development policy and programming.

In Germany, too, this development has picked up momentum in recent years. The German Ministry for Economic Cooperation and Development (BMZ), which coordinates German development cooperation, formally endorsed the role of rigorous impact evaluation in its 2021 evaluation guidelines (available in German here). It is also funding a research and advisory project run by the German Institute for Development Evaluation (DEval) to increase the application of rigorous impact evaluation methods and the use of evidence. In parallel, Germany’s two primary implementing agencies for development cooperation, GIZ and KfW, are exploring ways to integrate evidence into their projects and processes. 

J-PAL Europe has been working with these actors to support the integration of evidence into their work as a thought partner and “evidence broker” by providing training as well as synthesising and sharing relevant evidence generated by our network. As part of our partnership with GIZ, we have developed a new training on evidence use, and we are currently working with GIZ to facilitate evidence uptake in the organisation's project design.

Acknowledging and addressing barriers to evidence use

Despite an increase in recent years, the number of rigorous impact evaluations in the portfolio of German development projects is low, and evidence from randomised evaluations is not consistently used in the design of projects or strategic decision-making. 

A 2022 report by DEval reviewing the use of evidence and rigorous impact evaluations in German development cooperation identified lack of time, methodological knowledge, and financial resources as key barriers to rigorously evaluating projects, but also highlighted that the (perceived) usefulness of doing so was unclear to decision-makers. Other barriers to evidence take-up include low priority given to evidence in decision-making and the scarce and unsystematic dissemination of findings, among others. Nevertheless, 94 percent of respondents to a survey of German development professionals presented in the report consider evidence useful to improve the effectiveness of development cooperation.

These insights emphasise the need for financial resources as well as measures to increase technical capacity to understand, translate and apply evidence.

To address the lack of dedicated funding, BMZ committed to financing up to nine rigorous impact evaluations by 2025 as part of an ongoing pilot project run by DEval. Funding applications could be submitted by implementing agencies together with partnered research institutions after a matchmaking process and were reviewed by a review board, to which J-PAL Europe Executive Director Anna Schrimpf contributed alongside DEval staff and other experts.

Partnering to broker evidence

To tackle barriers to a more systematic use of rigorous evidence, GIZ and J-PAL Europe have deepened their partnership previously focused on training in evidence generation using impact evaluation to include activities structured around three pillars: 1) a training for GIZ staff in using and applying evidence, 2) a synthesis of existing rigorous evidence on selected approaches relevant to GIZ’s work, and 3) direct support to sector units in applying findings from this evidence to the design of new development projects.

Spread over three sessions, the evidence-use training included a discussion of what (rigorous) evidence is, how to find and assess the quality of evidence, and how to think about the generalisability of research results to different contexts. It also discussed organisational processes to ensure the iterative use of evidence throughout the project lifecycle. As part of the second and third pillars, we are now working with GIZ’s Evaluation unit and sector teams to identify evidence needs and explore ways to feed relevant evidence into the process of project design. 

The importance of an evidence broker 

At J-PAL, one of our key roles is to translate research into actionable policy recommendations. The need for organisations such as J-PAL to play this role as an “evidence broker” has been emphasised repeatedly in our conversations with policy partners in Germany and beyond. Decision-makers may not have the time and or technical skills to read research papers or work through and distinguish between the many different summary products that are available, including summaries of single evaluations, meta analyses, systematic or non-systematic reviews. 

Relatedly, project planners in development agencies emphasise the need for concrete and actionable recommendations, including suggestions for successful implementation. Researchers may be reluctant to provide such clear-cut recommendations for specific contexts based on available evidence without testing the approach in the new context first. 

Our partnership with GIZ offers an opportunity to learn more about how to best structure the role of evidence broker for an implementing agency in international development cooperation. Some of the open questions we hope to explore together include when in the project lifecycle evidence is most likely to be taken up, in which format it should be presented, and how to make it more actionable without over-generalising available research results. 

Another area we hope to explore is the institutional set-up that can allow evidence to be used more systematically in the long term. This involves the question of which audience within partner organisations can then integrate lessons from research and feed them into the organisation’s processes more widely, as well as thinking about the role of external partners such as J-PAL in a potential longer-term partnership around the use of evidence. Building on J-PAL’s experience working with government partners, like in Spain or in countries across Latin America, we look forward to exploring these questions together with our partners in Germany and to apply lessons learned to J-PAL Europe’s growing work with development actors. 

In addition to our work promoting the generation and use of rigorous evidence in domestic social policy-making, J-PAL Europe works with development actors across the continent to embed more evidence in development cooperation. We work with development funders and implementers to help share and synthesise relevant evidence, and to build capacity and institutional mechanisms for generating and using evidence widely. Learn more about our work on our website.