How government, academic, and nonprofit partners are working to evaluate a home visiting program in South Carolina to improve family health and wellbeing

Researchers from J-PAL’s network partnered with the South Carolina Department of Health and Human Services (SC DHHS) and the National Service Office for Nurse-Family Partnership and Child First on a rigorous evaluation of the Nurse-Family Partnership (NFP) program in South Carolina. As research on the program’s impact on birth outcomes concludes, we sat down to debrief with Jordan Desai, chief of quality at SC DHHS; William (Bill) Thorland, senior research fellow at the National Service Office for Nurse-Family Partnership and Child First; and Margaret (Maggie) McConnell, lead researcher on the study and associate professor of global health economics at Harvard T.H. Chan School of Public Health. Together, they bring us behind the scenes of the research partnership, discuss what challenges and opportunities they’ve experienced, and highlight what they look forward to learning as the study progresses.

Why was SC DHHS and the National Service Office for Nurse-Family Partnership and Child First interested in conducting a randomized evaluation of the NFP program in South Carolina? 

Bill: Conducting rigorous evaluations of NFP has been essential to the program’s success for the past five decades. In embarking on the first US Pay for Success project on maternal and child health, we acknowledged the need to test this new innovative financing mechanism and rapid scaling with a randomized evaluation. 

Jordan: Approximately 60 percent of the births in the state of South Carolina are covered through SC DHHS’ Healthy Connections Medicaid program. Like many other states, SC DHHS is interested in looking for data-driven innovations that can improve birth outcomes for the residents we serve. This project’s randomized evaluation was appealing to the agency because it offered new insight into the effectiveness of specific interventions.

Maggie, why were you interested in working with NFP and SC DHHS on this evaluation? How is this study contributing to the existing evidence base on interventions to support families during pregnancy and early childhood?

Maggie: In the United States, low-income families face significant challenges during pregnancy and early childhood that set the stage for lifelong inequities. Our research team recognized the value of learning about a flagship federally-funded program’s impact on outcomes in this population at this critical time. Our early findings underscore growing evidence that improving birth outcomes with behavior-change focused interventions during pregnancy can be challenging and that innovative thinking is needed to address the substantial inequities we see in the study. 

How did the research partnership come about? What was exciting, as well as challenging, about partnering on this evaluation?

Maggie: Initial conversations about the project started nearly a decade ago! Launching a complex project with many partners and sustaining engagement with the work over this long period required tremendous commitment from all partners but especially NFP and the state of South Carolina. Their commitment to building evidence enabled the launch of the largest randomized evaluation of nurse home visiting to date.

Bill: We were excited to partner on the largest randomized controlled trial of NFP and serve more families in South Carolina. Being able to target a large enough sample of the population to enroll in the trial was challenging for enrollment but essential to being able to detect statistical differences. It also afforded a unique opportunity to assess program effectiveness during a time when significant changes were occurring, such as the rapid decline in teenage birth rates and the expansion of community services and public health initiatives focused on maternal-child health.

SC DHHS was one of the first states to use a 1915b Medicaid waiver mechanism with an embedded randomized trial. What is this waiver and why is it important for future innovation and learning?

Jordan: Generally, 1915b waivers provide states with flexibility in how services can be tailored to specific population needs within a managed care delivery system. This waiver authority provides states with important flexibility by allowing them freedom to pursue innovative projects targeted toward specific populations.

Maggie: In the context of this project, with the support of a network of partners and philanthropic donors, South Carolina demonstrated the feasibility of embedding a randomized evaluation into a 1915b waiver. Through this integration, the state is a national leader and provides a pathway forward for other states to rigorously evaluate the impacts of their innovation.

The first academic paper of the study⁠—focusing on pregnancy, birth, and postpartum outcomes⁠—was just published. How do you foresee these results informing the NFP model and policies to support pregnant individuals?

Bill: Previous internal evaluations of NFP national data showed the potential for program impact on some measures of birth outcomes among specific sub-populations of families being served by Nurse-Family Partnership. Additional analysis among families served in South Carolina is needed to better understand how the families and conditions in the state during this time may have differed from what has been previously seen nationally. Ultimately, this informs decisions for more targeted enrollment practices and community-specific outcome expectations to guide the implementation of the program. This study further shows us we need to provide better support for nurses to reach and serve those families facing the most complex challenges and barriers to resources.

This study will also enable us to learn more about how NFP affects a range of life outcomes in several areas of life over time. What additional outcomes do you look forward to learning about as the study progresses? 

Jordan: SC DHHS is interested in learning about longer-term outcomes for the families involved in the study. The population served in this study, new mothers and children, makes up the majority of the state’s Medicaid population. As such, SC DHHS is particularly interested in data that could help the state better understand additional social determinants of health needs for this specific population.

Maggie: In the first few years of analysis, our research will focus on outcomes that are experienced during pregnancy, postpartum, infancy, and toddlerhood that can be observed in administrative data records. Our research team looks forward to exploring longer-term outcomes, such as educational and economic attainment, which may provide a more nuanced picture of how program participation changed families’ trajectories. The State of South Carolina is a national leader in its ability to link data at the state level for research, which will enable ongoing learning from the study even long after study participants have graduated from NFP’s services.   

Bill: We appreciate J-PAL’s commitment to more rigorous evaluations, and we look forward to seeing other findings that can inform additional opportunities to improve the program.  

What advice would you give to other organizations considering conducting a randomized evaluation of their policy or program?

Bill: Large, randomized evaluations can bring challenges. The rapid enrollment period placed a lot of stress on the nurses and took away from the resources needed to better enroll families who could most benefit. It can also be particularly difficult for providers, like the NFP nurses, to know there are potential families who could benefit, but are assigned to the control group. Nevertheless, we believe evaluations are essential to ensuring your program is learning and adapting to the changing needs of the population you intend to serve.