J-PAL North America’s US Health Care Delivery Initiative: Six reflections after six years
When I launched the US Health Care Delivery Initiative (HCDI) in 2013, my goal was to bring more rigorous evidence to bear on how to make the US health care system more efficient, effective, and equitable.
This goal was motivated by my findings with Sarah Taubman in Science that too few studies of health care delivery programs or approaches were empirically rigorous: less than 20 percent of studies in health care delivery were randomized controlled trials (RCTs), compared to more than 80 percent of studies of US medical interventions.
Since HCDI’s launch, thanks to generous support from Arnold Ventures and the Robert Wood Johnson Foundation, we have catalyzed many RCTs in health care delivery—often cheaper, faster, and larger-scale than many thought possible.
And now we’re at an exciting point in our journey when we have new, high-quality evidence on formerly unanswered questions in US health care.
This week, we are rolling out an HCDI blog series to share stories about RCTs in health care delivery conducted by world-class academic researchers in our network and funded by HCDI. These stories demonstrate how such studies are not only possible but also incredibly valuable in informing health care policy in the US.
We will be featuring a new post in the HCDI blog series each day this week, so stay tuned for more!
To kick us off, I wanted to share six reflections on how this emerging body of evidence underscores the value of RCTs in US health care delivery:
1. RCTs clearly and credibly identify a program’s impact.
Observational studies have found that workplace wellness programs could save money while improving employee productivity and health. However, Damon Jones, David Molitor, and Julian Reif’s recent RCT showed that a university-based wellness program had no impact on health and employment outcomes after two years.
And Zirui Song and Katherine Baicker’s RCT of a workplace wellness program reached similar conclusions.
Their studies are a great example of how random assignment ensures we measure the impact of a program, rather than underlying factors.
In the words of Molitor, Jones, and Reif, “results from non-RCT studies may be due to differences in who participates [in wellness programs], not differences caused by participating in the program.”
Read more on the workplace wellness study in "A Reason to Be Skeptical of Workplace Wellness Programs' Impact."
Does being of the same race as your doctor affect your health decisions? Answering important questions like this one by looking at historical data is difficult because many different factors other than race could be at play.
Research by medical doctors Marcella Alsan and Owen Garrick with Grant Graziani underscores how randomization can be used to study what you want to study rather than what nature or administrative data gives you.
In their study—an interdisciplinary effort driven by clinical researchers—they randomly assigned Black men to be seen by Black or non-Black physicians, allowing them to rigorously test whether a doctor’s race affects Black men’s demand for preventive health services.
Read more about this provider race concordance research in "Texas Tech agreement to abandon race in med school admissions will worsen health disparities."
3. RCTs can reveal the subtle effects of an intervention.
Policymakers and economists often think about external effects of policies and programs, beyond the impact on the people directly covered by the policy. In some cases, RCTs allow you to directly measure such spillover effects, which is difficult to do with other types of studies.
For example, RCT findings from Adam Sacarny and co-authors demonstrated that sending strongly worded, peer comparison letters to high-prescribers of an antipsychotic drug significantly reduced prescribing without evidence of harming patients.
He then further leveraged the data from the RCT to study if the letters had spillover effects on prescribers who worked with those who received a letter. Other researchers in J-PAL’s network are analyzing the spillover effects of other policies and programs.
Read more on this high-prescriber research in "New Evidence on Stemming Low-Value Prescribing."
4. RCTs can inform critical policy decisions at scale.
In my work, I often hear from people concerned that RCTs may be ill-equipped to test questions at large scale.
However, this is not the case. RCTs can teach us about the impacts of large-scale policies.
Dr. Darshak Sanghavi, the former director of the Preventive and Population Health Care Models Group at the Center for Medicare and Medicaid Innovation (CMMI), was able to launch the first ever RCT of a major federal health insurance program in the United States.
Read more in Dr. Sanghavi’s blog post "How do we achieve affordable, quality health care? Follow the evidence."
And Darshak’s story isn’t the only one. I’ve also co-authored an analysis of a randomized study of a nationwide payment reform program.
As we argued in our Science article, such large-scale RCTs are possible and should be closer to the norm than the exception.
5. More RCTs are underway that will teach us important lessons on the impact of food as medicine, medical debt, opioid reduction strategies, and more.
At J-PAL North America, we’ve seen how these results and those from many other RCTs can shape important health policy debates.
We’re excited to be supporting a number of ongoing RCTs, including a food-as-medicine intervention for food-insecure individuals with diabetes, a program that cancels people’s medical debt, an opioid buyback program, and many more.
These studies will shed new light on strategies to improve health care delivery in the US.
6. This work is only the beginning.
Our experiences to date demonstrate that it’s both necessary and possible to generate rigorous, policy-relevant evidence.
Our work has helped build a new foundation of rigorous evidence, but more work is needed. We must build on our momentum by continuing to generate more actionable evidence to improve peoples’ lives.
Stay tuned for a new post every day this week in our HCDI blog series. Interested in staying up to date with J-PAL North America and the Health Care Delivery Initiative? Sign up for our monthly newsletters here.
