Announcing J-PAL’s Science for Progress Initiative

As economists we hope to do work that matters. We hold each other to high standards for rigor, while at the same time pursuing relevance: insights that will make a real difference in people’s lives. Over the last 20 years J-PAL has epitomized this dual commitment, championing the use of experimental methods across a growing range of applied fields while hosting initiatives that address specific challenges—from agricultural technology adoption to governance to climate change to gender and economic agency—to which it has contributed timely, policy-relevant research.

One area where this combination of rigor and relevance has proven difficult to achieve is, ironically, in the study of the scientific process itself. Economists certainly appreciate that research is fundamental to economic growth and to human well-being more generally. And both governments and philanthropies invest heavily in supporting scientific research as a public good.

But we know too little about how to invest that capital effectively. What contracts, incentives, and institutions work best when funding scientific research? How should funders decide which projects to support? How much weight should be given to the proposal versus the track record of the proposer, for example? How can we get the benefits of expert peer review without throttling new ideas that are outside dominant research paradigms? And how can we ensure that the most talented individuals—including younger researchers, those entering science from non-traditional life and career paths, and more broadly members of traditionally under-represented groups—are not discouraged from pursuing science? On important questions like these, little systematic evidence exists.

The new Science for Progress Initiative (SfPI) aims to change this. It is part of a broader effort to equip the scientific ecosystem with the capabilities needed to apply the scientific method to itself—to create, in other words, a practice of “metascience.” Under SfPI, J-PAL will apply the methodological approach of randomized evaluation to the practice of funding scientific research. 

This effort has been a long time in the making. When Pierre Azoulay wrote in Nature in 2012 that we should “turn the scientific method on ourselves” he worried that this vision “will sound utopian to some.” But now various currents are converging and there is real momentum—and real support—to make it happen. SfPI aims to catalyze this, leveraging generous financial support from Open Philanthropy, Schmidt Futures, and the Alfred P. Sloan Foundation.

SfPI is launching at a moment of meaningful progress in the agenda of using research to improve the scientific process, led by two key US public institutions: the National Science Foundation (NSF) and the US Patent and Trademark Office (USPTO). For example, the USPTO recently announced a major investment in a new experimental art unit which will routinely conduct randomized evaluations with the goal of continuously discovering and scaling successful practices within the agency. SfPI Co-Chair Heidi Williams and J-PAL North America Scientific Director Lawrence Katz are part of a small team of academics supporting the new NSF Technology, Innovation and Partnerships (TIP) directorate’s Regional Innovation Engines program, aiming to use research to inform program design and surface potential opportunities for research and evaluation. And adding to this momentum is the fact that some private philanthropies, including the Gordon and Betty Moore Foundation, are making substantial investments in using data, evidence, and experimentation to improve and refine how they support science. 

The heart of SfPI is collaboration with organizations that fund and support science. In order to support and encourage more progress in this agenda, J-PAL is working in collaboration with the Institute for Progress (IFP), a nonpartisan think tank in Washington DC with an aligned mission and vision. Together with the Federation of American Scientists, IFP is launching a Metascience Working Group aimed at fielding, sourcing, and curating insights from the metascience community to share with participating federal agency officials and philanthropic science funders.

We (SfPI’s co-chairs, Paul Niehaus of UC San Diego and Heidi Williams of Dartmouth College) are also directly investing in efforts to encourage research on important but neglected research topics such as the management of science. In collaboration with Pierre Azoulay (MIT), Matt Clancy (Open Philanthropy), Patrick Collison (Stripe), Raffaella Sadun (Harvard Business School), Daniela Scur (Cornell), John Van Reenen (LSE), and Caleb Watney (Institute for Progress) we are planning a large-scale survey of management practices in US-based biomedical research labs. 

A separate team of faculty from Harvard Business School (HBS)—Rem Koning, Karim Lakhani, and SfPI invited researcher Kyle Myers—are, with support from a gift to HBS from an anonymous alumni donor, developing a management training program which they will study in a randomized evaluation dovetailing on this survey, aimed at better understanding how to improve the management and organization of scientific labs. 

The SfPI advisory committee, which will guide funding and other decisions, includes leaders from across a wide spectrum including the academic, philanthropic, public and private sectors. These include Matt Clancy (Open Philanthropy), Patrick Collison (Stripe), Dan Correa (Federation of American Scientists), Tyler Cowen (George Mason University), Kumar Garg (Schmidt Futures), Daniel Goroff (Sloan Foundation), Bishakha Mona (Science Philanthropy Alliance), Emily Oehlsen (Open Philanthropy), Elaine Sevier (Research Theory), and Caleb Watney (Institute for Progress). 

And we want to hear from more researchers and potential collaborators. This is a nascent area of scientific inquiry, with few people currently doing the type of work we envision supporting. We welcome experimental researchers who want to leverage their skills by applying them to the scientific process. We welcome domain experts who understand the economics of science and innovation and want to learn and implement experimental methods. And we welcome relationship builders: forging strong, trusted relationships with institutional partners has been and will be the foundation of the whole enterprise.

To that end, SfPI is creating an unusually broad and flexible array of ways in which researchers can engage:

1. We can respond to time-sensitive opportunities. Partner institutions often have a short window of opportunity during which they can say “yes” to a collaboration. We want to help support researchers in seizing those opportunities. We will welcome off-cycle and exploratory proposals at any time, using a streamlined application process, and we commit to turning around these proposals within two weeks.
2. We want to hear ideas from researchers and potential collaborators. In addition to J-PAL affiliates who are eligible to apply by default, we are building a large group of invited researchers with domain expertise who are eligible to apply to SfPI. Other non-invited or not-yet-invited researchers are welcome to reach out to us to explore matchmaking opportunities.  
3. We will work with a wide range of experts. We are making it as easy as we can for non-experimentalists to run their first experiments. While our proposal form asks about the key technical features of the proposed experiment, these do not need to be perfected in order for you to submit an initial proposal. We have the ability to route proposals that need stronger technical development to the Research Management Support unit with J-PAL North America to provide support on aspects of experimental design and implementation, with the cost covered by SfPI.

Finally, we want to hear about your good ideas that still don’t quite fit into our frameworks. We expect to learn as we go and iterate on the models that we use to allocate resources. The point of SfPI is, after all, to learn together how to accelerate science for the social good.

To learn more, visit the SfPI webpage, read the press release, or contact us.