A federal office to bolster evidence-based policymaking

Rigorous evidence is an essential tool for identifying effective solutions to social problems. In 2015, the federal government of the United States—with staffing support from multiple research organizations in the behavioral science field including J-PAL North America—formally established the Office of Evaluation Sciences (OES) within the General Services Administration (GSA) to help agencies build and use evidence. Since that time, OES has leveraged the expertise of 140 multidisciplinary researchers to conduct over ninety evaluations with twenty federal agencies and impacted policymaking across a wide range of sectors.

The Problem

Federal policies aim to serve Americans effectively and utilize their tax dollars efficiently, yet without rigorous evidence it is challenging to determine if policies are meeting their intended goals or, if not, how they can be improved upon. 

Public policies seek to address pressing social problems for the benefit of individuals, families, and society at large by distributing tax funds into various federal, state, and local programs. In doing so, policymakers aim to achieve two goals: 1) ensure programs are as effective and accessible as possible and 2) minimize costs for American taxpayers. Rigorously evaluating social programs can help policymakers achieve both goals by identifying innovative solutions rooted in behavioral science. 

Behavioral science utilizes economic and psychological research to better understand how people engage with, participate in, and respond to policies and programs. For example, government agencies can utilize insights from behavioral science to determine how to encourage more low-income Americans to access benefits to which they are entitled. By evaluating programs and applying the resulting insights, the federal government can deliver better results at lower costs. 

The Policy Solution

In 2015, the Obama Administration established the White House Social and Behavioral Sciences Team (SBST) and the General Services Administration founded the Office of Evaluation Sciences (OES) for the purpose of bringing evaluation and evidence use to agencies across the federal government.

The Obama Administration noted the potential of behavioral insights for maximizing the effectiveness of government programs. In 2013, the White House convened a multidisciplinary team of experts, including representatives from J-PAL North America, to develop a plan for institutionalizing behavioral research and evidence in policymaking across all aspects of the federal government. Through temporary assignments¹ within the U.S. government, J-PAL North America staff² supported the SBST and helped establish OES.  

In its first year, SBST studied various federal programs, finding numerous opportunities to strengthen these programs and identifying successful, behaviorally-informed solutions to many of their problems. Following the release of its first annual report in 2015, President Obama formally established SBST within the White House Office of Science and Technology. A second organization, OES, was also established within the GSA to sustain and expand evidence-building capacity government-wide.

Today, OES continues to execute its mission to “build and use evidence to better serve the public.” To achieve this mission, OES utilizes a six-step process (see Figure 1) for conducting evaluations and is guided by the following core principles: rigor, relevance, transparency, independence, and ethical practice.

“Innovative academic partnerships have been crucial to establishing the Office of Evaluation Sciences and central to us meeting the important milestone of 100 completed collaborations with agency partners to date.” - Kelly Bidwell 

From Research to Action

OES has not only conducted over ninety evaluations with more than twenty government agencies but has also, with J-PAL North America’s technical assistance and staffing support, developed resources to help agencies meet new evaluation requirements per the Foundations for Evidence-Based Policymaking Act of 2018. 

Since its establishment in 2015, OES has continued to expand the use of evidence-based policymaking by conducting rigorous evaluations and providing resources to government agencies. The organization is currently running twenty studies on a wide range of programs. Studies have been conducted in partnership with a range of agencies, including but not limited to the US Department of Labor, Department of Education, Health and Human Services, Housing and Urban Development, and the United States Agency for International Development. Once evaluations are completed, OES helps these agencies apply the findings into concrete, cost-effective policy. As a result, these studies and subsequent policy changes have touched millions of lives, both in the United States and abroad. 

The multidisciplinary team at OES employs and collaborates with academic researchers in various fields to best support the needs of agency partners. For example, OES and the Centers for Medicare and Medicaid Services (CMS) have partnered with J-PAL affiliate Adam Sacarny (Columbia University) on two randomized evaluations to examine the effectiveness of sending letters to doctors who overprescribe certain medications. In a 2015 study co-authored by J-PAL North America Co-Scientific Director Amy Finkelstein (Massachusetts Institute of Technology) and others, Dr. Sacarny researched whether sending peer-comparison letters to potential overprescribers of opioids and other controlled substances with high potential for abuse would reduce inappropriate prescribing. Results showed that these letters did not have an effect on prescription rates, leading researchers to think creatively about new ways of reaching these prescribers. 

In Dr. Sacarny’s second OES study, the research team identified 5,055 high prescribers of quetiapine, an antipsychotic that is often prescribed outside its approved indications, and sent them three strongly-worded letters that highlighted their prescription rates compared to peer physicians. Findings demonstrate that the letters reduced prescribing. This result has the potential to reach not only the over 2,500 physicians in the treatment group and their nearly 44,000 patients, but also thousands of other physicians and patients unnecessarily prescribing or taking quetiapine. By reducing overprescribing, these simple, low-cost letters also provide a tool to reduce unnecessary Medicaid and Medicare spending.

Following evaluation results, OES helps government agencies understand and utilize findings to create evidence-based policies and practices. For instance, OES worked with several agencies to conduct a series of ten related studies on improving enrollment in retirement plans, finding that even small barriers could limit enrollment. Results from these studies were shared openly to promote cross-agency learning and led to increased enrollment, with millions of additional dollars contributed to retirement accounts across federal agencies. In 2017, this evidence base informed the National Defense Authorization Act, which instituted automatic retirement contributions for service members. 

In addition to conducting evaluations directly, OES aims to enhance the capacity of government agencies to build and utilize rigorous evidence themselves. They provide training and resources on general evaluation practices and on implementing the Foundations for Evidence-Based Policymaking Act of 2018, or “Evidence Act.” The Evidence Act requires all federal agencies to create learning agendas, produce Annual Evaluation Plans, and hire Evaluation Officers (EOs), further institutionalizing evidence use across the government. To aid agencies in this process, OES created toolkits, in addition to presentations and trainings created in collaboration with J-PAL North America. In 2019, a senior policy associate from J-PAL North America worked as a fellow with OES and the Office of Management and Budget (OMB) to develop onboarding materials for new EOs with varied levels of expertise in evaluation. J-PAL North America’s training team has facilitated one such onboarding session on impact evaluation methodologies, bolstering the evaluation capabilities of approximately twenty EOs in attendance.

As OES grows its evaluations and reach, J-PAL North America and its researcher network continue to lend expertise in pursuit of further expanding evidence-based policymaking. For more information on OES or to apply to its fellowship program, please visit oes.gsa.gov/opps.