Charter schools and student learning

There has been wide variation in the estimated impacts across charter schools. The average per year effect of attending a charter school in the sample of 113 schools was an increase of 0.080 standard deviations in math and 0.046 standard deviations in English language arts. However, the estimated effects for individual schools varied widely. At some charter schools, students who won the lottery and attended the charter school had substantially higher test scores than students who lost the lottery and attended a traditional public school. At other charter schools, students who won the lottery actually performed worse than students who lost.An open question for researchers and policymakers is why charter schools with negative effects on test scores are oversubscribed and whether, over time, demand increases at schools with positive effects or decreases at schools with negative effects.

In Massachusetts, lottery winners for charter schools located in urban areas often did substantially better than lottery losers. Lottery winners for charter schools in nonurban areas fared, on average, about the same or somewhat worse compared to lottery losers. The figure below shows that, in Massachusetts [3], middle school charter applicants in both urban and nonurban areas had low proficiency rates prior to applying. Over time, performance for lottery winners who attended urban charter schools improved substantially, while performance for lottery losers improved modestly in English language arts and actually declined in math. In contrast, performance for lottery winners who attended nonurban charter schools declined over time, while performance for lottery losers improved.

One possible explanation for this pattern is that, in Massachusetts, students who lost lotteries for urban charter schools tended to attend traditional public schools with below average test scores, while students who lost lotteries for nonurban charter schools tended to attend traditional public schools with above average test scores. In the studies included in this summary, the impact of attending a charter school is estimated by comparing the experience of students who won the lottery for a particular charter school to the experience of students who lost the lottery for that school. As a result, a charter school that attracts students who, after losing the lottery, attend poor-performing traditional public schools will be estimated to have a more positive impact than an otherwise identical charter school that attracts students who, after losing the lottery, attend better-performing traditional public schools.

A common feature of the charter schools with the most positive effects has been the adoption of a No Excuses educational approach. In a sample combining lottery-based estimates of charter school impacts from New York City [4], Massachusetts [3], and the national charter school study [5], each of five school practice characteristics aligned with the No Excuses approach—frequent teacher feedback, the use of data to guide instruction, increased instructional time, high dosage tutoring, and high expectations—was positively associated with charter school effectiveness in at least one subject. In contrast, school resource inputs—class size, per pupil expenditures, the fraction of teachers with an advanced degree, and the fraction of teachers with a teaching certification—were not predictive of charter school effects in math. In English language arts, higher per pupil expenditures were associated with more effective charter schools, but smaller class sizes were associated with less effective charter schools.

It is unclear whether No Excuses charter schools have tended to have the most positive effects because of their No Excuses approach, or because these schools often locate in the most disadvantaged urban areas. As noted earlier, charter schools in urban areas, where the traditional public schools have been generally poor performing, have had more positive effects than charter schools in nonurban areas, where the traditional public schools have been generally better performing. Charter schools in urban areas have tended to be No Excuses schools, while there have been very few No Excuses schools in nonurban areas.

After taking into account both urban location and the performance levels of the fallback traditional public schools, and controlling for other school characteristics, none of the school practices were associated with higher test scores in English language arts, and intensive tutoring was the only No Excuses school practice that was associated with higher test scores in math. This evidence suggesting the potential effectiveness of intensive tutoring is in line with several recent randomized evaluations that found large increases in student performance from tutoring [6] [7] [8] [9] [10].

In urban areas, the effect of charter school attendance was larger for black students, Hispanic students, and previously poor-performing students. Combining data from the Massachusetts[3] and national charter school studies [5], urban charter schools had positive effects across most groups of students, with the exception of white students, for whom the effect of charter school attendance was marginally significant in math but not statistically significant in English language arts. In general, urban charter schools had the most positive effects on students who were less advantaged, including black and Hispanic students, those with low baseline test scores, those receiving subsidized lunch, and English language learners. The effect of urban charter school attendance on test scores was similar for both students who were in special education and those who were not.

Nonurban charter schools had negative effects across most groups of students, including female students, white students, and students without low baseline test scores, not receiving subsidized lunch, not in special education, or not English language learners. The effects of nonurban charter schools were marginally positive for black or Hispanic students and those receiving subsidized lunch.

The table below shows the estimated impact of a year of charter school attendance across different subgroups of students, using data from the Massachusetts [3] and national charter school studies [5].

Some No Excuses charter schools in urban areas have been shown to have positive impacts on longer-term outcomes. While most studies of charter schools have looked at the impact of charter school attendance on test scores, a smaller number of studies have examined longer-term impacts, including improving college preparation and enrollment, reducing teen pregnancy, and reducing incarceration among male students.

In New York City [11], students offered admission to the Harlem Children’s Zone Promise Academy middle school had higher rates of enrollment in any college immediately following high school graduation and higher rates of immediate enrollment in a four-year college.Pregnancy rates were 10.1 percentage points lower among lottery winners, relative to a pregnancy rate of 17 percent among those who lost the charter school lottery, and incarceration rates for males dropped from 4 percent among those who lost the lottery to zero among lottery winners [11]. In Boston [12], attendance at a charter high school increased pass rates on the state’s graduation exam, SAT scores, advanced placement exam test taking, and advanced placement exam scores.While charter school attendance did not have a significant effect on college enrollment rates, it shifted enrollment from two-year to four-year colleges. Among students whom researchers could follow for at least 18 months after graduation, charter attendance reduced immediate enrollment in a two-year college by 11 percentage points (relative to an average enrollment rate of 19 percent among students who lost charter school lotteries) and increased immediate enrollment in a four-year college by 18 percentage points (relative to an average enrollment rate of 41 percent among students who lost charter school lotteries) [12].

A study in Texas that randomly assigned traditional public schools to implement a set of practices associated with the most effective charter schools found that these practices increased math achievement. In Houston, Texas [7], researchers tested whether the five school practice characteristics that were associated with the most effective charter schools in New York City—increased learning time, more effective teachers and principals, more student-level differentiation through tutoring or computerized instruction, frequent use of data to alter the scope and sequence of classroom instruction, and a culture of high expectations—could increase student achievement in traditional public schools. Researchers randomly assigned sixteen low-performing traditional public elementary schools to either a treatment group, which implemented the set of school practices, or a control group. On average, attending a school that implemented the set of school practices increased test scores by 0.112 standard deviations per year in math but did not have a statistically significant impact on test scores in reading.

Charter schools in disadvantaged urban neighborhoods can generate large test score gains for students admitted via lottery, as well as corresponding improvements in longer-term outcomes. In general, urban charter schools have had the most positive effects on students who are least advantaged, including black and Hispanic students, those with low baseline test scores, those receiving subsidized lunch, and English language learners. Studies of charter schools in Boston [12]and New York City[11] found that charter school attendance also increased immediate enrollment in four-year colleges and reduced teen pregnancy and male incarceration. In contrast, nonurban charter schools have had no or even negative effects on test scores, on average.

Urban charter schools adopting a No Excuses approach have been associated with the largest gains in academic performance. Most of the urban charter schools included in the lottery-based studies reviewed here have adopted a No Excuses approach characterized by strict and clear disciplinary policies, mandated intensive tutoring, longer instruction times, frequent teacher feedback, and high expectations for students. In combined data from Massachusetts [3], New York City [4], and national studies [5], this set of No Excuses practices was positively associated with charter school effectiveness.

It is hard to disentangle the influence of locating a charter school in a disadvantaged urban neighborhood from the impact of adopting No Excuses practices. One reason for the large gains associated with No Excuses schools is that these schools tend to locate in urban areas where the traditional public school alternatives are very poor performing. However, the policy conclusion that charter schools located in low-performing districts that adopt a No Excuses approach have produced positive impacts remains clear.

Evidence suggests that implementing mandated, intensive tutoring is one potential strategy for improving urban schools. Using data from Massachusetts [3], intensive tutoring was the only school practice characteristic significantly associated with charter school effectiveness in math, once urban status and the performance of fallback traditional public schools is taken into account. In Houston [7], researchers implemented five school practices associated with the most effective charter schools in low-performing traditional public schools; however, due to funding constraints, intensive tutoring was implemented in only math and only in one grade per school. Among the middle and high schools in the treatment group, students in grades that received tutoring had math test score gains of 0.608 standard deviations per year, compared to 0.208 standard deviations for students in grades that did not receive tutoring. In Chicago, a randomized evaluation found large gains in math test scores from implementing intensive tutoring in twelve traditional public high schools [9].