Effects of Replicating Charter Schools in Boston, Massachusetts
- Students
- School administrators
- Enrollment and attendance
- Student learning
- Provider Performance
- Administrative reform
- School-based inputs
Scaling effective education programs is one strategy to address the achievement gap in the United States and elsewhere. However, the success of academic programs in one location often falls short upon expansion, in part due to the difficulty of replicating factors such as effective teachers, school leadership, and peer environments. Researchers studied a policy reform in Boston, Massachusetts, that allowed effective charter schools to replicate their school models at new locations. Results showed that charter schools reproduced their effectiveness at new campuses by following similar sets of standardized protocols, demonstrating the feasibility of charter school replication.
Policy issue
Scaling effective education programs is one strategy to address the achievement gap in the United States and elsewhere. However, the success of academic programs in one location often falls short upon expansion, in part due to the difficulty of replicating factors such as effective teachers, school leadership, and peer environments. As charter schools also tend to serve as testing grounds for innovations in education, understanding their ability to scale successful practices is critical. Despite these challenges, recent evidence shows that policies of “No Excuses” charter schools – including high expectations, strict discipline, and data-driven instruction – may improve educational outcomes when implemented in traditional public schools or low-performing schools converted to charter status. Can schools that boost student outcomes reproduce their achievement gains at new campuses?
Context of the evaluation
In 2010, the state of Massachusetts passed the Act Relative to the Achievement Gap, a law to reform and expand the charter school sector. The new policy increased the state’s cap on charter school funding, and allowed for the expansion and replication of “proven providers” – or schools considered to be most effective by the Massachusetts Board of Elementary and Secondary Education – for their track records of success. In the four years after the act’s passage, the number of Boston charter schools doubled from 16 to 32, and some existing charters expanded to new grade levels. Among these schools, most new campuses were expansions of existing “No Excuses” charters.
Charter enrollment in Boston expanded significantly after the reform, from 15 percent of sixth graders in 2010 to 31 percent in 2015. Charter enrollment also increased among elementary and high schools, though not as substantially. The increase in demand was greatest among charter schools that were neither proven providers nor expansions.
In terms of pedagogy, most Boston charter schools follow the “No Excuses” model, which features high expectations for academics and behavior, selective teacher hiring, extended instructional time, and more. Research has shown that Boston charters increase math and English standardized test scores, as well as longer-term outcomes, including SAT scores, Advanced Placement credit, and college enrollment. Compared to traditional schools in the Boston Public Schools (BPS) district, charter schools offer more time in the classroom, including Saturday school and school break tutoring programs. Charter schools tend to employ younger and less experienced teachers, who are less likely to be licensed.
Applicants to proven provider locations and expansion locations differ across several characteristics, and from BPS students. Before the 2010 Act, about 56 percent of applicants to all charter schools were Black, compared to 42 percent of BPS students. After the act, charters tended to attract a more disadvantaged, lower-achieving population than their original proven provider locations. Applicants to expansion charters scored below the BPS average on fourth-grade math and English tests and were more likely to be eligible for subsidized lunches, a pattern likely reflecting changes in recruitment practices after the 2010 Act, which required all charters to enroll higher-need students.
Details of the intervention
Researchers evaluated changes in the effectiveness of Boston’s charter middle school sector during a period of rapid expansion by examining charter school admission lotteries from 2004 to 2013. The lottery records covered 14 of 15 charter schools admitting students into fifth or sixth grade, which accounted for 94 percent of enrollment for schools in this category during the 2013-2014 school year. In total, researchers analyzed the outcomes of 3,188 student applicants to the charter lottery prior to the 2010-2011 expansion, and 7,139 applicants afterward.
Researchers used state administrative data from 2002 to 2014 to match information on students’ test scores and other educational outcomes with their lottery records. This included data from both charter lottery applicants and other Boston students. They also used administrative data to track changes in charter application and enrollment trends following the 2010 reform.
Results and policy lessons
The policy reform allowed effective charter schools to replicate their models and generate achievement gains on par with those from original “proven provider” locations, while continuing to improve student outcomes at original campuses.
Impact on test scores: Students who won the lottery to attend a charter school increased their math and English test scores relative to students who lost the lottery. One year of attendance at a Boston charter middle school boosted math scores by 0.18 to 0.37 standard deviations, and increased English Language Arts scores by up to 0.23 standard deviations. Attendance at proven providers and expansion charters led to the highest achievement gains.
Impact on enrollment: Students offered admission to a charter school via lottery increased their charter school enrollment by about one year before the 2010 reform, and around six months after the reform. Researchers suggest the difference in enrollment gains is likely due to having only three years of data on post-reform enrollment, and a potential lag between students receiving a lottery offer and choosing to enroll in a charter.
Ability to replicate charters: Proven providers and other charters maintained their effectiveness after the charter expansion reform of 2010. Similar to the pre-reform period, a year of attendance at a proven provider or other charter school boosted math scores by 0.37 or 0.19 standard deviations on average, and English scores by 0.19 or 0.13 standard deviations. This suggests that expanding to operate new campuses did not detract from practices at schools existing before the 2010 Act. Proven providers also successfully replicated their pedagogical practices and impact from original locations to expansion charters after the reform, as evidenced by expansion charters generating comparable achievement gains to their parent campuses. The highly standardized practices in original locations may have facilitated the scaling process.
Overall, results show that expansion charters produced large increases in student test scores and achievement, comparable to their parent campuses. This demonstrates the feasibility of charter school replication.
The study's findings were public and circulated prior to the 2016 Massachusetts ballot question, which proposed allowing further charter school expansion. The question did not pass.