Deworming: An Informed Debate Requires a Careful Look at the Data

By Esther Duflo and Dean Karlan

Transparent sharing of data and analyses is crucial to good science and informs sound policy. The two organizations that we helped found, the Abdul Latif Jameel Poverty Action Lab (J-PAL) and Innovations for Poverty Action (IPA) are committed to, and actively support this kind of public sharing of data ourselves and hope to see more growth in research transparency.

The release of two papers (here and here) last month re-analyzing data made available by our affiliates Michael Kremer (Harvard) and Edward Miguel (UC Berkeley) from their seminal 2004 study, and a response paper by Hicks, Miguel and Kremer, reopened a debate on the educational impacts of school-based deworming, which IPA and J-PAL have supported.

The first paper, a “pure” replication of the original work, found the same main conclusions as the 2004 paper: school-based deworming leads to increased school participation. The second paper was, however, a re-analysis of the original study, which used different assumptions that led to different results. These results have prompted a media and social media storm and widespread questioning of the original policy conclusions.

Kremer and Miguel, however, contend that many of the key assumptions made in this re-analysis are incorrect, and stand by their original findings. Many other prominent development experts including Chris Blattman (Columbia University, who, for full disclosure, is a J-PAL and IPA affiliate) and Berk Ozler (World Bank), also disagree with many of the underlying assumptions used in the re-analysis.

We wholeheartedly support the effort to re-analyze data to uncover possible coding errors, as well as the sensitivity of results to critical assumptions. As Paul Gertler (UC Berkeley, a J-PAL and IPA affiliate also not involved in the original research) has said, however, it is important that “replications should be held to the same standards as any other study”. With any study, a different set of assumptions will naturally deliver different results. However the superiority (or at least equal validity) of any new set of assumptions needs to be justified before this invalidates the original results. Since the original authors, and several other experts, contest the assumptions of the re-analysis, it seems premature to conclude that the original study conclusions are invalid before the debate is settled.

Given the importance of this evidence for school-based deworming policies, which has already reached over 95 million children, and given how involved IPA and J-PAL have been throughout the deworming research and scale-up policies, IPA and J-PAL would like to encourage an independent, technically focused, re-appraisal of both the study and its re-analysis.

As organizations striving to ensure that policy is informed by rigorous evidence, we hope that other researchers, not affiliated with J-PAL or IPA, will take on this re-appraisal. We have reached out to Emmanuel Jimenez, Executive Director of 3ie, which funded the re-analysis through its replication program, and he wrote to us that, “given the substantive scholarly exchange on the original paper and the re-analysis, 3ie would be happy to continue supporting further work in this area.” We look forward to collaborating with 3ie and other researchers.

This debate also underscores the value of multiple replications of the original study on school based deworming using randomized evaluations in different countries where worms are endemic. We would welcome such well designed and executed studies on the impact of school based deworming on educational outcomes.

Based on their own review of the evidence (including the original paper, the re-analyses, the Cochrane review, and other studies), other organizations (Evidence Action, Givewell, Center for Global Development) have decided to continue to support deworming in general and school-based deworming in particular. While we await the results of the re-appraisal of the Kremer and Miguel data, IPA and J-PAL will also continue to support efforts at school based deworming.

Esther Duflo is the Abdul Latif Jameel Professor of Poverty Alleviation and Development Economics at the Massachusetts Institute of Technology and a Director of J-PAL and Dean Karlan is a Professor of Economics at Yale University and President of IPA.