Achieving Academic Success After School: A Randomized Evaluation of the Higher Achievement Program in the United States
- Students
- Empowerment
- Student learning
- Training
The complexity and slow nature of improving schools with below-average learning outcomes have compelled many policymakers and parents to consider educational opportunities outside of the school system, such as out-of-school (OST) academic enrichment programs. Researchers evaluated the academic impact of a “Cadillac” OST program in Washington, DC. Researchers found that the program increased students’ problem-solving and reading comprehension scores after two years.
الموضوع الأساسي
Improving the quality of educational instruction has the potential to increase children's test scores, graduation rates, and subsequent earnings. However, many children living in historically-disinvested neighborhoods attend under-funded, lower-performing public schools. While improving these schools is a desirable goal, the complexity of school reform and the slow nature of such change have compelled many policymakers and parents to consider educational interventions outside of the school system, such as out-of-school (OST) academic enrichment programs. Academically focused OST programs, in particular, aim to improve students' academic attitudes, behaviors and performance by increasing youths' access to high-quality academic supports and opportunities. But these programs range widely in their focus, structure and intensity variations that likely affect their ability to make such improvements. Indeed, the other rigorous evaluations of after school programs have not found strong positive impacts on academics.
This multi-year study focuses on investigating the impact of a "Cadillac" version of academic enrichment OST programs by analyzing the effects of the Higher Achievement Program, a mature OST program with a rigorous and intensive academic curriculum. If this comprehensive program fails to generate long-term improvements in academic outcomes, then OST programs in general may not be capable of doing so, but if it succeeds, then it will demonstrate that such programs can have lasting impacts on students' lives.
سياق التقييم
The D.C. public school system struggles with a number of structural and functional problems and as a result has failed to adequately educate numerous students. With 59 percent of students receiving a high school diploma in 2001, Washington D.C. held the 4th lowest graduation rate in the nation.1 D.C., like other struggling school systems in the country, has been experimenting with reforms; closing under-populated schools, fixing up decrepit buildings, sweeping out underperforming principals, and hiring new young teachers. Yet if school reforms are ultimately effective, it may be a number of years before changes in student learning are seen. OST academic enrichment programs such as the Higher Achievement Program may supplement student learning in the meantime.
معلومات تفصيلية عن التدخل
Higher Achievement is an established OST program that strives to develop positive academic behaviors and attitudes in academically motivated but underserved fifth- through eighth-grade youth. Its objective is to improve participants' grades, standardized test scores, and school attendance in order to increase their educational opportunities, specifically through acceptance and scholarships to attend private, parochial, and public magnet high school programs.
The program is extremely time-intensive, offering approximately 650 hours a year of academic instruction, enrichment activities and mentoring during the after-school and summer hours. During the school year, participants, or "scholars," attend the "After-School Academy" three days a week. This 25-week program, which runs from 3:30 to 8:00 p.m., includes homework help, dinner, an elective, a 15-minute group meeting, and two hours of academic instruction in small groups of about two to three scholars one day a week in mathematics, one day in literature, and one in technology. Volunteer mentors lead these groups. Scholars participate in monthly field trips and community service projects. During the summer, the six-week "Summer Academy" operates from 8:00 a.m. to 4:00 p.m., five days a week. Students attend four classes a day, taught by trained faculty, in mathematics, science, social studies, and literature, as well as two electives. Scholars take weekly field trips and participate in a three-day university trip during which they experience college life by attending classes, sleeping in dorms, going to lectures, and eating in dining halls. In both the After-School and Summer Academies, students receive high school placement services.
The intensity of the program and HA's strict admission policies screen for motivated parents and what the program defines as "academically motivated" students. While academic performance is not a criteria for admission to the program, it is of course possible that such families may succeed academically even without access to the HA program. To identify the effects of attending HA above and beyond what these families would have achieved without the program, we built the study around an oversubscription randomized controlled trial.
During recruitment for the study, Higher Achievement recruited more youths than were required to fill the available positions; slots were then randomly allocated among eligible students. Students were evaluated using a short version of the Stanford Achievement Test and a student survey containing psychometric scales designed to assess the effects of the program on students' perceptions of themselves, their peers, and the options available to them in the future. Parents also complete brief surveys.
النتائج والدروس المستفادة بشأن السياسات
Test Scores: The program had little effect in the first year, but by the second year, student scores had improved by 0.09 standard deviations in reading comprehension and 0.12 standard deviations in problem solving.
Academic Attitudes: Surprisingly, rather than improving students' perceptions of their academic abilities, the program seems to have negatively impacted students' academic attitudes in the first year. However, by the second year, there are no differences between the treatment and comparison students' attitudes. These changes in attitude may have been driven by the transition into a more competitive environment. Students entering the Higher Achievement program in fifth grade experience a decline in academic confidence upon entrance. Students in the comparison group have relatively higher confidence as fifth graders, but then experience a similar decline as they make the transition to middle school in sixth grade.
Behavior: Participation in the program increases the rate at which children report engaging in many forms of misconduct. For example, the probability of taking something that did not belong to the child increases by 14 percentage points in the first year and 19 in the second year. Though this could reflect an increase in self-reporting, rather than an increase in actual misconduct, this is clearly an important area for future investigation.
High School Preparation: Students in the program were more likely to visit a high school, get information about a high school, talk with non-parental adults and peers about high schools, and decide where to apply.
Change over the summer: Academic skills do not increase over the summer, relative to the comparison group. High school preferences, however, do show a marked change. In the spring, 9 percentage points more students in the treatment group express a desire to attend a competitive area high school compared to the control group. In the fall, this increases to 17 percentage points, and 10 percentage points fewer students express a desire to attend their local public school.