Leveraging Evaluation and Evidence for Equitable Recovery (LEVER) Evaluation Incubator

The LEVER Evaluation Incubator welcomes applications from US state and local government agencies interested in evaluating programs and policies that advance economic mobility and racial equity in their communities. See a list of past and current J-PAL North America State and Local Evaluation Incubator partners here. 

Applicants must be open to working with J-PAL North America to look for ways to include random assignment in the implementation of the program or policy they are interested in evaluating. For example, slots for a program with a waitlist might be allocated by a lottery; eligible beneficiaries of an undersubscribed program may be randomly encouraged to sign up; multiple versions of a program may be randomly assigned to individuals to test them against one another, and pilot programs or programs being expanded to new locations could provide opportunities to randomly select who receives the program first. 

While each engagement looks different, we find that agencies with the following are typically the most prepared for the Incubator: 

• A specific policy or program you are interested in evaluating. Programs and policies that tend to be conducive to randomized evaluation include pilot programs, new programs before scale-up, programs introducing a new component, and programs expanding to a new population; new policies being implemented; and relatively new policy initiatives where data are needed to assess impact. 
• An interest in using randomized evaluation to assess the impact of a program or policy
• A general idea of how randomization could work for your program and inform policy or program improvement. 
• An initial understanding of the priority outcomes your team wishes to evaluate. 
• The number of people targeted by a policy, or number of participants eligible for the program, including those who do and do not currently receive the program. 
• Motivation to build on established relationships to plan an evaluation that will substantively engage community residents, groups, and/or organizations.

After you submit your letter of interest, we will schedule a coaching call to discuss your submission in more detail and identify areas of support in preparation for submitting your full proposal. In addition to this support, you can:

• Sign up to attend office hours;

• Review our past Incubator informational webinar; 

• Reference our blogs featuring research-practice partnerships in government;

• Refer to our research resources on randomized evaluation;

• Reach out to [email protected] with any remaining questions. 

In phase 1, Letters of Interest will be reviewed by J-PAL North America staff. In phase 2, a Review Board composed of J-PAL affiliated researchers Bruce Sacerdote and Day Manoli, as well as Vincent Quan, co-executive director of J-PAL North America, will review full proposals.

While LEVER’s Evaluation Incubators empower agencies to take action on a specific evaluation idea, Training Sprints will equip agencies to build internal capacity, culture, and processes to develop sustainable evaluation practices, paving the way for future evidence use and creation.

During the spring 2024 Training Sprint, 10-15 teams from different jurisdictions around the country will be selected to learn together in weekly virtual sessions across ten weeks. The Training Sprint will help state and local government leaders incorporate best practices for evidence-based decision-making into their current processes so that the recovery funding evidence and evaluation standards for policy solutions become a long term practice. ​​Participants will:

• Develop their own policy for how evidence, evaluation and equity are prioritized in their jurisdictions. 

• Build a decision-making framework that makes their policy actionable and sustainable so that they consistently incorporate equity, evidence use and evaluations into their practices and initiatives. 

J-PAL North America specializes in randomized evaluation, the most rigorous type of impact evaluation. Randomized evaluation can determine the causal impact of a program by comparing the outcomes of a group that receives a program (individuals, communities,  schools, etc.) against a group that does not. Because individuals are randomly assigned to either the program or comparison group, we can be confident that the individuals do not differ systematically at the beginning of the study. We can therefore confidently attribute any difference in outcomes after the program to the intervention itself.

Treasury’s Economic Recovery learning agenda has identified the importance of understanding  short-term and long-term impact on households, organizations, communities, and governments from specific American Rescue Plan State and Local Fiscal Recovery Funds (SLFRF) projects in priority policy areas such as affordable housing, workforce, and public safety. Randomized evaluation is a critical tool that states and localities can use to advance knowledge and inform future decision-making in these areas.

Community engagement in research and evaluation encompasses a range of approaches for working with and involving residents, leaders, groups/organizations, and stakeholders along the research continuum, from conceptualization to dissemination and application. Engaged research is designed to ensure projects reflect the perspectives, strengths, and priorities of community members. Ideally, projects are built on mutually beneficial and reciprocal relationships that center community leadership in decision-making, therefore providing a structure for inclusive and equitable assessments, analyses, and policy recommendations. Given the complexity and dynamic nature of communities, we are looking for projects that intentionally define what constitutes ‘community’ and ‘engagement’ in the context of an evaluation project.

By nurturing mutually beneficial, reciprocal relationships grounded in trust, a community engaged approach can contribute to the reliability of research processes and data collection. This, in turn, enhances internal and external validity, leading to more robust research with increased community buy-in and equity.

By collaborating with community members on evaluation projects, states and localities can:

• Ensure equitable representation of people on the frontlines who are directly impacted by programs and policy decisions. 

• Bring rigor to the research by developing questions, analyses, and data collection strategies that accurately reflect community dynamics and are responsive to community priorities. 

For additional guidance, please reference the following: J-PAL blog post on the value of centering lived experience in the research process, Urban Institute’s Community Engagement Resource Center, and Durham University’s guide to ethical principles and practices in Community Based Participatory Research.