The Medium-Term Impacts of High-Achieving Charter Schools in the United States
- Students
- Urban population
- Dropout and graduation
- Student learning
- School choice
High-performing charter schools have been shown to improve test scores of low-income students in cities. However, it is unclear whether test score gains have an impact on longer term academic performance, risky behaviors, or health outcomes. Researchers examined results of a school admissions lottery to measure the impact of charter school education on students after six years. They found that, in addition to improving test scores, offering a charter education resulted in higher college enrollment, some reduction in specific risky behaviors, and no physical or mental health effects.
Policy issue
The use of charter schools as a way to improve public education for underserved students is widely disputed. Public charter schools are publicly funded schools run autonomously by independent organizations. They are exempted from certain education regulations and rules, but are held accountable by results-based standards (outlined in a “charter”) that focus on student achievement. Proponents believe charter schools’ alternative approaches, such as extended school days, to be an effective means of providing children facing barriers to academic success with educational resources in a supportive academic setting. Critics often find fault with the “teach to the test” and paternalistic environment associated with charter schools’ emphasis on data-driven metrics and results.
While there is no clear evidence suggesting that a typical charter school is any more effective at increasing test scores than a typical public school, there is an emerging body of research suggesting that high-achieving charter schools can improve the lives of low-income students in cities. Yet few studies focus specifically on high-performing charter schools, and even fewer examine the longer-term outcomes of receiving such an education.
Context of the evaluation
Promise Academy is a 1,500 student charter school located in Harlem, New York City.
It is part of a larger 20-program Harlem Children’s Zone designed to address the myriad social and economic problems that low-income children in New York City face.
Promise Academy offers students after-school tutoring, medical services, healthy meals, and parental support. The school offers an extended school day, and some students attend classes on Saturdays as well. The average Academy student spends nearly 1,800 hours in school per year – around 46 percent more time than a New York City public school student. Promise Academy uses a data-driven approach to retain only the highest-achieving teachers, who in turn stress a “no excuses” culture of achievement, with college attendance as the default expectation for its graduates.
Details of the intervention
Due to high demand for limited spots, Promise Academy conducted lotteries for sixth grade admissions in 2005 and 2006. This randomized selection process provided researchers with the opportunity to evaluate the specific effect of an Academy education (or lack thereof) on students.
Six years later, in 2012, researchers contacted lottery participants as they concluded either 11th or 12th grade. In total, researchers surveyed 150 lottery winners and 257 non-winners. In addition to conducting standardized math and reading tests, researchers asked about students’ educational achievement and attainment, risky behaviors, and health outcomes. They also gathered administrative data on high school course-taking and college enrollment.
Researchers also used administrative data to compare the outcomes of Promise Academy students to those of students at other high-achieving charter schools across the nation.
Results and policy lessons
Researchers found that the offer of a Promise Academy education did result in positive longer-term effects, but impact varied significantly across outcomes.
Academic achievement: The Academy’s focus on academic achievement did sometimes translate into reality. On standardized tests, lottery winners generally scored higher. Lottery winners were 13 percentage points more likely to graduate from high school in four years, relative to a 58 percent average among the comparison group. However, there was no difference among graduation likelihood over a six-year period, meaning that lottery winners may not actually benefit from overall higher graduation rates. A similar pattern held for college enrollment, with higher immediate college enrollment among lottery winners, but no overall difference in likelihood to ever enroll in college between winners and non-winners.
Risky Behavior: Results were mixed across types of risky behavior. Reported teen pregnancy rates among female lottery winners were 10 percentage points lower than the rate of 17 percent among non-winners. For males, incarceration rates dropped from 4 percent among non-winners to zero for winners. Researchers found no differences in reported rates of drug use, alcohol use, or criminal behavior.
Health: Despite winners reporting healthier eating habits than non-winners, there were no discernable health effects between the two groups. Participants reported similar physical and mental health across a variety of categories.
Overall findings suggest that there is a causal relationship between improved test scores and some medium-term outcomes, and that high-achieving charter schools like Promise Academy can positively affect both.