Reflecting on a decade of impactful research at J-PAL North America: Forming strong researcher-practitioner partnerships
In part three of J-PAL North America’s ten-year anniversary blog series, we dive into research partnerships and look at the evidence from Summer Youth Employment Programs and Creating Moves to Opportunity to distill key lessons on the effective collaboration between researchers and practitioners needed to build an evidence base on poverty reduction strategies.
When a study’s design reflects questions that both researchers and policymakers are asking, it is well-positioned to identify and understand a program’s underlying mechanisms. What’s more, when strong researcher-practitioner partnerships are in place, evidence is more likely to have greater policy impact as government decision-makers and non-profit leaders work with researchers to put findings into practice.
J-PAL North America has supported two areas of policy-relevant research over the past ten years that serve as prime examples of what strong researcher-practitioner partnerships can accomplish. The Creating Moves to Opportunity (CMTO) project is an ongoing collaboration between J-PAL affiliated researchers and public housing authorities to introduce and evaluate programs designed to help families who receive Housing Choice Vouchers exercise housing choice and move to neighborhoods with greater economic opportunity. Likewise, rigorous research by J-PAL affiliates in partnership with city agencies in Boston, Chicago, New York, and Philadelphia has shown that participating in municipal summer jobs programs, often called Summer Youth Employment Programs (SYEPs), can have many benefits for youth both during and beyond the summer.
Building research projects based on the aligned interests of researchers and practitioners from inception is crucial
It’s no secret that randomized evaluations can take substantial time, resources, and commitment from all involved. For the work of a randomized evaluation to be sustainable and policy relevant, it’s important to find a research idea that is meaningful to the implementing organization and can also answer questions of academic interest. CMTO, for example, was built from the aligned interests of housing stability practitioners and researchers, who played equal roles in developing the intervention and shaping the research. The study was designed to determine whether to scale a new program (a pressing question for housing authorities) and to better understand what drives residential segregation by income and race in the United States (an important question for the academic team).
In the case of J-PAL affiliates Sara Heller (University of Michigan) and Judd Kessler’s (University of Pennsylvania) work with New York City’s Department of Youth and Community Development (DYCD), the commitment of program staff to evaluate their SYEP led to an iterative process of refining research questions that were meaningful for all involved. Speaking about the value of this process, Sara Heller noted, “A group of staff who have a real interest in learning and improving is perhaps the single most important thing for researchers to accomplish policy-relevant research. The staff’s commitment to learning and self-reflection leads to great communication, allows us to understand the relevant institutional details, and helps us improve our ideas.”
Building aligned interests requires deep trust. Partner organizations need to know that researchers understand their context and value their priorities, which are often different from those of researchers. For Sara Heller, this often means providing technical assistance on topics that would never become a research paper in order to establish a shared language and understand the ways the partner institution works. She shared that researchers “may have to give up on some great ideas that just prove too complicated for your partners, or that they aren’t interested in. You have to learn to be flexible and compromise.”
It can be a great risk for government agencies and non-profit organizations to take on an evaluation. Studies are investments that can produce results decision makers were not expecting or hoping for. It’s key for researchers to be able to come into an evaluation on the terms of the partner and be willing to break down power imbalances between the research team and implementing partner. When a researcher speaks a shared language with a partner, they’re better able to bridge the gaps between differing priorities and conflicting timelines in order to build this trust.
The members of the CMTO project worked hard to develop a shared language with many different areas of expertise represented on the team, including sociology, Housing Choice Voucher program operations, housing policy, microeconomics, communications, and more. In a 2019 blog post, their research team reflected, “Many other research projects involve a similar number of stakeholders, but what has made this project particularly unique has been the degree of communication and coordination across the different partners, which has enabled the team to maximize the contributions of each perspective.”
Trusting researcher-practitioner partnerships are key to evidence-based policymaking
When shared language is established and goals are aligned, trusted partnerships between researchers and practitioners can lead to critical evidence-based policymaking. In the SYEP work, this meant significant program expansion based on research results in both Chicago and New York.
However, this trust doesn’t just mean that government partners are more likely to scale up or scale down based on findings. Trusting partnership can also lead to innovative approaches to program design and delivery that meets community needs.
For example, for Sara Heller, Judd Kessler, and Julia Breitman of New York’s DYCD, this trust led to an iterative research and program development process that resulted in a pilot program component where youth formally request letters of recommendation from their summer job supervisors. Judd Kessler reflected, “The letters project involved contacting supervisors and youth ourselves, and that was something that DYCD trusted us to do after seeing the work that we had done in our prior research.” Based on these results, DYCD and the research team are in conversation about rolling out letters of recommendation to many more of the nearly 100,000 annual SYEP participants in New York.
J-PAL North America is building a suite of resources and services to cultivate strong research partnerships
Based on learnings of how to best support research partnerships, J-PAL North America developed Evaluation Incubators to facilitate trusting engagements between researchers and practitioners. Over the past ten years, we’ve supported over fifty organizations and agencies in this work. By connecting partner organizations and agencies to researchers, leading robust training in evaluation practice through our annual Evaluating Social Programs course, and providing tailored technical assistance, we work to develop shared language and goal alignment as community-based organizations and government agencies turn policy-relevant research ideas into randomized evaluation designs.
In 2015, we also established the State and Local Innovation Initiative, recognizing that state and local government agencies play a significant role in the work of building rigorous evidence on strategies for reducing poverty. Since then, through convenings, Evaluation Incubators, and now through the Leveraging Evidence and Evaluation for Equitable Recovery (LEVER) program, J-PAL North America remains committed to building the capacity of state and local governments to work with researchers in order to run rigorous, policy-relevant evaluations. We know that the results of such research—when paired with trusting researcher-practitioner partnerships—have the potential to influence policies, programs, and communities for generations to come.
In J-PAL North America’s ten-year anniversary blog series, we reflect on some of the most impactful randomized evaluations and bodies of research that our organization has supported over the past decade. We also celebrate the tremendous contributions of our researcher network and the policymakers and practitioners who have made this research possible. Part one kicks off the series with reflections from our scientific leadership. Part two explores the role of study design and implementation. Part four discusses how credible evidence can identify effective strategies to reduce poverty, regardless of the impact estimate. Part five dives into how policy can be informed at scale.
