Individualized tutoring to improve learning

Individualized tutoring can help students who have fallen behind in grade level catch up. Research by J-PAL affiliates, Jon Guryan (Northwestern University), and Jens Ludwig (University of Chicago), and co-authors Roseanna Ander, Phil Cook, Kenneth Dodge, Roland Fryer, Susan Mayer, Harold Pollack, and Laurence Steinberg, found that Saga Education’s intensive math tutoring program helped male high school students facing multiple barriers in Chicago perform better in school. The results suggest that it is possible to substantially improve academic outcomes for students facing multiple barriers even after they reach adolescence. Following the evaluation, Saga Education has expanded its individualized tutoring program to serve 12,000 students in New York City, Chicago, and Washington, DC. Saga Education is advising the implementation of a similar model in three cities in the Netherlands.

The Problem

In many US classrooms, particularly those in under-resourced communities, students enter high school several years behind grade level and struggle to catch up, putting them at high risk of dropping out.

Students may struggle to develop the critical foundational skills necessary to learn more advanced concepts. Subsequently, they may enter high school several grade levels behind, particularly in math. For example, 30 percent of American 8th graders scored below basic proficiency in math, according to the 2017 National Assessment of Educational Progress (NAEP—commonly referred to as the United States’s national report card).¹ Despite the more than US$600 billion the United States spends in K–12 public schooling annually, many classrooms are not well equipped to help students who have fallen behind catch up.²

This challenge is exacerbated by the frequent mismatch between what is taught in the classroom and a student’s level of knowledge. In some cases, students who struggle with foundational concepts (for example, division) are taught advanced concepts (such as exponents) that are mandated by the core curriculum. When students have not acquired the prerequisite skills, they can experience significant difficulties learning new material that builds on earlier concepts, furthering the gap between students who are below grade level and their peers.

This gap in learning is particularly pronounced between white and minority students. Over the past decade, white students scored, on average, 0.8 standard deviations above black students on the NAEP—a test score gap roughly equivalent to what the typical American teenager learns between 8th grade and the end of high school. Economists estimate that an improvement in test scores by 0.8 standard deviations would be projected to mean a change in annual earnings of 22 percent, based on a recent Program for the International Assessment of Adult Competencies (PIAAC) survey. Compared to their white peers, Black and Hispanic youth are also about 60 percent more likely to drop out of high school. Many education policymakers and practitioners believe that by the time low-income students enter high school, the effects of poverty are too deeply entrenched for teachers and schools to make a significant difference.³

Even when teachers seek to support students who have fallen behind, the wide variation in learning levels within a single classroom and the pressure to teach a core curriculum makes it challenging to tailor instruction to the specific needs of every student. Yet research from around the world suggests that targeting instruction to a student’s learning level—rather than the curriculum specified by their grade level—can lead to large gains in student learning, especially in classrooms with significant differences in student achievement.

The Research

Two randomized evaluations found that students who received individualized math tutoring through Saga scored higher on exams, earned better grades, and were more likely to pass high school classes.

During the 2013–2014 school year, J-PAL affiliates and co-authors partnered with Chicago Public Schools to study the impact of individualized math tutoring on academic outcomes for 9th and 10th grade male students in twelve public high schools in some of Chicago’s lowest-income communities. The average student in the program had a GPA of 2.1 out of 4.0 and had missed approximately one month’s worth of school in the previous school year. One in five study participants had been arrested before the study began.

Saga assigned students to a one-hour tutoring session every day as part of their regular class schedule. Tutors met with two students at a time and divided instructional time evenly between reviewing foundational skills—targeting instruction based on the needs of individual students—and working on current topics from students’ regular math classes. Students received credit for tutoring, which replaced either a second period of math that many Chicago Public School students typically take, or an elective course.

The evaluation found profound effects on young men’s academic achievement. Students in Saga learned an extra one to two years’ worth of math beyond what their peers learned in an academic year. Tutoring raised participants’ average national percentile rank on 9th and 10th grade math exams by more than 20 percent. GPAs increased by 0.58 out of a 4.0 grade point scale, and the students’ pass rates in math increased by more than 50 percent.

For more information, see the evaluation summary.

From Research to Action:

Evidence on Saga’s impact has helped catalyze expansion to other major urban areas and secure philanthropic funding.

Match Education, a Boston-based charter school network, first developed this tutoring model in 2004, implementing the program in its charter schools. In 2014, Match leadership spun off to form Saga Education, a nonprofit organization that has expanded this intensive tutoring program to public schools in major urban areas.

Saga Education has leveraged rigorous evidence of its impact to scale up in Chicago Public Schools and introduce the program in the New York City School District. “Without the RCT, it would not be possible to scale aggressively,” said Alan Safran, President and Chair of Saga Education, when asked of the impact of randomized evaluation on the tutoring program.

If we can [expand this tutoring approach] for the school and for the city and around the country, we are going to have a lot more kids be successful.

— United States Secretary of Education Arne Duncan, on expansion of Saga Education

Following Saga’s presentation of results from the Chicago-based randomized evaluation, the New York City Mayor’s Office decided to allocate US$6 million over three years to implement the program in New York City public high schools and middle schools. Saga Education also leveraged the results of the study to make successful cases for significant grants from the Laura and John Arnold Foundation, the Chan Zuckerberg Initiative, and Chicago and New York City foundations to expand the program. After these initial investments, school districts themselves have used their federal Title I funding to sustainably support Saga’s operations. 

In 2019, Saga received US$2.2 million in AmeriCorps funding for Saga’s programs in Chicago, New York City, and Washington, DC. This funding allowed Saga Education to employ 150 AmeriCorps tutors across the three cities in 25 high schools, to serve an additional 2,800 students. In total, Saga has reached over 12,000 students across three cities in its first five years alone.

This case study was published in October 2018 and updated in April 2021 to include figures from 2019-2020 as well as expansion to additional cities.

References

Cook, Philip J., Kenneth Dodge, George Farkas, Roland G. Fryer, Jonathan Guryan, Jens Ludwig, Susan Mayer, Harold Pollack, and Laurence Steinberg. "The (Surprising) Efficacy of Academic and Behavioral Intervention with Disadvantaged Youth: Results from a Randomized Experiment in Chicago." NBER Working Paper No. 19862, January 2014. https://doi.org/10.3386/w19862.

Cook, Philip J., Kenneth Dodge, George Farkas, Roland G. Fryer, Jr., Jonathan Guryan, Jens Ludwig, and Susan Mayer. "Not Too Late: Improving Academic Outcomes for Disadvantaged Youth." Northwestern University Institute for Policy Research Working Paper, February 2015. https://www.ipr.northwestern.edu/documents/working-papers/2015/IPR-WP-15-01.pdf

Ander, Roseanna, Jonathan Guryan, and Jens Ludwig. "Improving Academic Outcomes for Disadvantaged Students: Scaling up Individualized Tutorials." The Hamilton Project Policy Proposal, March 2016.

Jonathan, Davis M.V., Jonathan Guryan, Kelly Hallberg, and Jens Ludwig. "The Economics of Scale-Up." NBER Working Paper No. 23925, October 2017. https://doi.org/10.3386/w23925.