December 2021 North America Newsletter

Good afternoon,

As we approach the end of 2021, I find myself reflecting on the simultaneous tension and promise of the past year. This year was filled with ongoing grief and evolving challenges from Covid-19, with the pandemic reaching its two-year mark. Yet it was also filled with innovative ideas and tremendous drive from so many to address urgent and longstanding needs of communities experiencing poverty in North America.

I am deeply grateful for the partnership of this incredible community of researchers, policymakers, funders, and practitioners. Your enduring commitment and action enabled and strengthened all of our work this year, as we collectively strive to generate essential research and harness evidence to inform policies and programs that improve lives. For example, in Puerto Rico, multiple agencies are working with J-PAL affiliated researchers and staff to explore topics ranging from computer-assisted learning to workforce development. In Boston, Economic Mobility Pathways (EMPath) is collaborating with researchers in J-PAL’s network and the Boston Housing Authority to evaluate a new version of a comprehensive economic mobility coaching program. 

These are just two examples. Several more are featured in our Celebrating RCT Partners campaign. Without them and countless others, our work this year would not have been possible.

In our December newsletter, we highlight three projects that rely on strong partnerships. Five new government partners are joining us via the State and Local Innovation Initiative to evaluate important policy questions. Working with public health officials in Contra Costa County, CA, J-PAL affiliates found behavioral nudges and financial incentives were not effective for increasing Covid-19 vaccine uptake, suggesting local governments and employers may need to explore other methods. Lastly, we share a case study on the Office of Evaluation Science’s work to institutionalize the generation and use of rigorous evidence among US federal agencies, which J-PAL North America has collaborated on since its inception.

Thank you, all, for your partnership. We look forward to continuing this critical work together in the year to come. 

With gratitude,
Vincent Quan
Co-Executive Director, J-PAL North America

Five new state and local governments launch partnerships with J-PAL North America

This month, J-PAL North America announced five new state and local government partners selected through the latest round of our State and Local Innovation Initiative. With support from J-PAL, the Colorado Department of Higher Education, Ohio’s Franklin County Department of Job and Family Services, the New Mexico Public Education Department, Puerto Rico’s Department of Economic Development and Commerce, and Oregon’s Jackson County Fire District 3 will explore randomized evaluations to generate widely applicable lessons critical to alleviating poverty in the United States. These partners will aim to evaluate policy-relevant questions on a variety of topics ranging from mitigating the risk of wildfires that disproportionately impact vulnerable communities to fostering upward mobility through case management services for job-seekers. We are excited to work with each of these partners on projects that have great potential to not only improve programs in their local contexts, but also provide a useful model for government agencies across the country facing similar challenges.

New research results: Financial incentives and behavioral nudges did not increase Covid-19 vaccine uptake among those who are vaccine hesitant

Millions of Americans remain unvaccinated against Covid-19. Many local governments and employers have attempted to use financial incentives and other behavioral nudges to increase vaccine uptake and combat vaccine hesitancy. However, a recent J-PAL-supported randomized evaluation by Mireille Jacobson (USC), Manisha Shah (UCLA), and coauthors found that financial incentives and other behavioral nudges did not meaningfully increase Covid-19 vaccination rates among members of a public Medicaid managed care plan in Contra Costa County, California. After thirty days, none of the behavioral nudges led to increases in vaccine uptake. For some groups, the nudges actually led to reduced vaccination rates—highlighting the potential for financial nudges to backfire.

Case study: A federal office to bolster evidence-based policymaking

In 2015, federal leaders established the Office of Evaluation Sciences (OES) within the General Services Administration to improve the capacity of US federal agencies to build and use rigorous evidence. Since then, OES has leveraged the expertise of 140 multidisciplinary researchers to conduct over ninety evaluations with twenty federal agencies and impacted policymaking across a wide range of policy areas. In a new case study on institutionalizing evidence use, we share the story of how J-PAL North America staff and affiliated professors contributed to the creation of OES and continue to partner with the federal office to aid their mission. As OES grows its evaluations and reach, J-PAL North America and its researcher network continue to lend expertise in pursuit of further expanding evidence-based policymaking.

Featured Evaluation Summary: Incentivizing Physical Activity to Improve Mental Health During Covid-19 Disruptions

Covid-19 has caused unprecedented lifestyle changes in the United States, altering patterns of physical and social activity, including a decrease in physical activity and increase in risk of anxiety and depression for many young adults. To better understand the association between physical activity and mental health, researchers conducted a randomized evaluation to measure the impact of financial incentives for reaching daily step goals on physical activity and depression risk. Program participants significantly increased their daily steps relative to the comparison group, though this difference dissipated when researchers removed incentives. Despite previously suggested links between physical and mental health, the intervention group saw no significant improvement in their mental well-being.

Featured Research Resource: Working with a third-party survey firm

Research teams may decide to contract a survey firm rather than run a survey in-house if the research organization has limited capacity or experience working with a given population, and there are established survey firms in the area. However, survey firms may not have experience working on randomized evaluations. This research resource provides guidance on when to use an external survey firm in a randomized evaluation and the process of identifying and contracting with a firm. It highlights topics in survey or study design that may be beneficial to review with the survey firm and describes measures to take in order to ensure high-quality data collection.