Research Incubator: Designing Randomised Evaluations for Humanitarian Interventions

J-PAL Europe is pleased to announce that applications are now open for a three-day evaluation design workshop tailored to the interests of humanitarian organisations that intend to explore randomised evaluations as a complement to their evaluation, learning and innovation toolkit. As long as public health guidelines allow, the workshop will be held in person in Paris, France.

Over the course of this research incubator, J-PAL researchers and staff will guide groups of 2-4 staff members per organisation through the process of designing a randomised impact evaluation. By the end of the incubator, participants can expect to have thought through the key elements of a rigorous evaluation design to better understand one of their own projects or interventions. As places are limited, early application is encouraged.

Successful applications will demonstrate that the potential learnings from the proposed evaluation subject are of high relevance to the organisation or the humanitarian sector more broadly. Applications from groups working on protection outcomes, such as effective protection programming in humanitarian interventions, the promotion of restraint among armed actors, or methods to better reach particularly vulnerable populations, will be preferred.

From mid-2022 onwards, and conditional on further funding, J-PAL intends to launch a competitive call for research proposals. Workshop participants will be in a unique position to work on the ingredients for a strong proposal with the input of J-PAL affiliated professors and senior staff members ahead of time.

Content and objectives

Together with  J-PAL affiliates and staff, participants will design a randomised impact evaluation of their own intervention through an integrated learning programme. Besides on-site teaching and group work, participants will build on preparatory online lectures, covering topics such as:

• Building a theory of change, selecting indicators and identifying data sources
• Methods of randomisation
• Performing power calculations to determine sample size
• Communication with stakeholders, including on ethical considerations

Participants will come out from the incubator with a robust understanding of the nuts and bolts of designing and implementing a randomised evaluation. Besides the design of an evaluation that is relevant to their work, they will be better equipped to make informed choices about evaluation options that are suited to their programmes and the specificities of humanitarian interventions.

What does a good application look like?

Successful applications will include:

• A clear outline of the evaluable activities within the overall project description. These could be:
  • Recurring activities of an ongoing programme, which participating organisations seek to better understand and iteratively improve
  • Activities of a programme that the organisation intends to launch in the near future
• Initial thoughts on the assumptions underpinning current programming and the research or evaluation question that the organisation seeks to answer with the support of a randomised evaluation
• Organisational interest in evidence-based decision making and learning that makes for a conducive environment for randomised evaluations
• A team composition that illustrates organisational interest and ability to carry randomised impact evaluations forward.
• Relevance of the expected learning for the organisation and the wider community of practitioners

Application resources

Randomised evaluations are a specific methodology to measure the impact of an intervention. If no one on your team has experience in working with randomised evaluations and/or followed an introductory course on this methodology, we invite you to consult the resources available on our website. Albeit not tailored to humanitarian contexts, we expect this set of videos to be particularly helpful to build your understanding of what a randomisation is and what it entails.

Preparing for the incubator

Should your application be selected, we will ask that all team members enroll in J-PAL’s upcoming online course on “Designing and Running Randomised Evaluation”, which will start on 1 February 2022. 

The course  introduces the basic concepts associated with randomised evaluations and will equip you to make the most out of your time with J-PAL’s researchers and staff. If you have already completed a similar course, we encourage you to browse through the different sessions by way of a refresher ahead of the incubator in March. Please note that enrollment in the course is open and free, but you may find it useful to set aside about half a day per week to complete the course work throughout February. 

Please note that we ask participant organisations to commit to sharing an evaluation concept note, building on the output of the incubator, with the course organisers by 24 March, two weeks following the completion of the workshop. If useful, we will share comments and feedback on the written document and otherwise use it to document the outcome of the incubator.

Indicative agenda

Key information

How to apply: Apply by downloading the application form, filling it out, and uploading it via this portal by 2 February. As places are limited and applications are reviewed on a rolling basis, we encourage you to apply as soon as possible.

Fees: Participation in the course is free of charge, thanks to funding provided by the UK Foreign, Commonwealth & Development Office (FCDO), We are, however, asking participants to cover their own expenses.

Travel and logistics: We intend to hold  the workshop in person in Paris, France, as long as public health guidelines allow; a final determination on the format will be made in mid-February. Please note that J-PAL Europe is not in a position to cover travel or accommodation expenses, and cannot provide visa letters to enter France.

Contact: Please contact Abdoulaye Ndiaye at [email protected] for any questions regarding your application or other details about the workshop. Please note that our office will be closed from 24 December to 3 January, 2022; you may then expect responses from 4 January 2022 onwards.

For more information on J-PAL's research effort on the effectiveness of humanitarian interventions, visit our website.

Application form

J-PAL Europe Humanitarian Research Incubator - Application Form