J-PAL North America joins the Learning Engineering Virtual Institute to improve math outcomes for middle school students

Covid-19 has severely impacted learning across the United States. School closures and other disruptions have led to significant test score declines, particularly among students from low-income households, those in schools with high poverty rates, and Black and Latino/a students. 

Math scores reflect some of the largest test score declines—this is of particular concern to policymakers as middle school math scores are a strong indicator of future academic success. Nationally, only 26 percent of all eighth-grade students scored proficient on the National Assessment of Educational Progress (NAEP) math assessment in 2022. The same year, thirteen percent of eighth-grade students who qualified for free or reduced price lunch scored proficient compared to 38 percent of students who didn’t qualify. This growing challenge requires evidence-based solutions that don’t place further burdens on an already strained education system.

The mission of the Learning Engineering Virtual Institute (LEVI) is to double the rate of middle school math learning within five years with a focus on students from low-income backgrounds. To achieve this mission, teams across the region are developing learning products that harness the potential of AI and machine learning. LEVI is an ongoing, multiyear collaboration across institutions and disciplines and we are thrilled to have joined this effort in October 2023. J-PAL North America is serving as the Evaluation Hub to prepare seven teams to launch randomized evaluations of their learning products. 

The seven teams engaged in LEVI—Carnegie Learning, Carnegie Mellon University, Eedi, Khan Academy, Rising Academies, the University of Colorado Boulder, and the University of Florida—will provide students and teachers around the world with improved tutoring services, intensive professional training, and other important support programs to advance middle school math achievement. Interventions range from an AI-powered chatbot that provides personalized math tutoring to generative AI video technology that digitally replicates the experience of a personal tutoring session.

As teams refine their learning products throughout the year, the Evaluation Hub will build the capacity of the LEVI teams to conduct rigorous impact evaluations in the long-term. Randomized evaluations make it possible to obtain a rigorous and unbiased estimate of the causal impact of an intervention; in other words, what specific changes to participants’ lives can be attributed to the program. If thoughtfully designed and implemented, a randomized evaluation can answer questions such as: How effective was this program? Were there unintended side-effects? Who benefited most? Which components of the program work or do not work? How cost-effective was the program? How does it compare to other programs designed to accomplish similar goals? 

To prepare all seven LEVI teams to conduct high-quality impact evaluations, J-PAL North America’s Evaluation Hub will provide technical assistance tailored to the unique needs of each organization. Additionally, J-PAL North America will host several workshops on topics such as identifying and selecting data; prerequisites to launching a randomized evaluation; implementation; and advanced randomization strategies. The Evaluation Hub will also facilitate connections with researchers in the J-PAL network and provide ongoing opportunities for shared learning as teams navigate the ever-changing landscape of AI. 

J-PAL North America is honored to be part of LEVI and to join them in their mission of improving the rate of middle school math learning. LEVI will significantly expand our understanding of how to best help students at scale through ed-tech products; connect the dots between research, policy, and practice; and ensure that learning products are moving the needle on key student outcomes. 

If you’re interested in learning more about this engagement, please contact Cat Darrow, J-PAL North America’s Associate Director of Research. For more information about LEVI, visit the LEVI webpage.