The Effect of Mentoring on School Attendance and Academic Outcomes: A Randomized Evaluation of the Check & Connect Program
- Students
- Enrollment and attendance
- Coaching and mentoring
Chronic absenteeism in early grades has been found to be predictive of high school dropout, but little is known about effective solutions to reduce absenteeism. Researchers conducted a randomized evaluation of Check & Connect (C&C), a school-based mentoring program, to evaluate its impact on school attendance and academic outcomes. The study was conducted in Chicago Public Schools (CPS) from 2011-2015. For students who began the program in grades 5-7, the program decreased student absences and caused participating students to fail fewer courses. However, there were no measurable effects of participating in the program for students who began the program in grades 1-4.
Policy issue
Despite the documented returns to high school graduation, many urban public school systems struggle with low high school graduation rates. The 10 largest school districts in the United States had graduation rates below 75 percent for the 2009-2010 school year. High school graduation is associated with a range of life outcomes, including crime involvement, health, and earnings1 . Several studies have shown that chronic absenteeism in early grades is predictive of high school dropout2 . Despite many schools having policies and programs to combat absenteeism, little experimental research has been done to test which of these approaches improves attendance. This study evaluates the impact of structured, long-term mentoring on attendance and academic outcomes for chronically absent elementary and middle school-aged students.
Context of the evaluation
This evaluation took place in the Chicago Public Schools (CPS), one of the largest public school systems in the United States. Among the schools selected to participate, the median free or reduced price lunch rate (a measure indicative of youth poverty levels) was 97.2 percent. Chicago, like many large, urban public school districts, struggles with low high school graduation rates. In 2010, the four-year graduation rate was 70 percent overall and just 57 percent for African American males. More than half of 10th-12th graders missed at least 10 percent of school days (the threshold for being considered chronically absent), and 13 percent of elementary students missed at least four weeks of school in the 2011-12 academic year.
The C&C program has been implemented in several other large school districts, including Minneapolis and San Diego. While randomized evaluations of C&C in high schools have shown mixed results, this study tests its efficacy with students in grades 1-7 who the researchers hypothesize may not yet be as credit deficient or disengaged as high school students.
Details of the intervention
In partnership with SGA Youth and Family Services, researchers conducted a randomized evaluation to test the impact of C&C on school attendance and academic outcomes for chronically absent students. C&C was introduced to CPS for the purposes of this evaluation. SGA hired the mentors as full-time employees and a project manager to manage and provide guidance to the mentors.
C&C pairs students at risk of disengagement with a mentor. The program has two main components: the “Check,” focused on monitoring students’ attendance and academic performance for signs of disengagement, and the “Connect,” personalized interventions provided by mentors to help students get back on track, including connecting them to school-based resources and social services. Though the program is relatively standardized, mentors are encouraged to tailor their support to what individual students need to overcome barriers to attendance. During this evaluation, mentors and students met an average of five times per month. Mentors also engage with students’ guardians through home visits or calls.
The study was run in two two-year cohorts, and the researchers used a three-stage process to randomly select which students would be offered the program. Students in grades 1-7 who had 10 to 35 absences the prior academic year were eligible to be selected for the program. For the first cohort, researchers selected 69 K-8 public schools in Chicago that were broadly representative of the school district and had enough chronically absent students. The selected schools were placed into groups of three by geography, demography, and school-level absence rates. One school in each group was randomly chosen to implement the C&C program. Within each school, the researchers randomly selected five grades out of grades 1-7 and then randomly selected one-third of the students in those grades to be offered the program. Selected students were offered the chance to participate in the program until mentors had full caseloads (30 students). A total of 487 students participated in the program in the first cohort, which ran for the 2011-12 and 2012-13 school years. An additional 2,958 students were randomly assigned to the control group and were not eligible to participate in the program during this period.
In the second cohort, nine of the 23 schools in the first cohort continued participating in the program. Researchers randomly selected some students who participated in the program in the first cohort to participate again, and some who were in control group in the first cohort to participate in the second cohort. Researchers also randomly selected some students to participate who had not been eligible for the first cohort (because they were new to the school or had been too young), as well as students who were nominated by the school’s principal. The second cohort ran for the 2013-14 and 2014-15 school years and included 348 students. The 1,111 students who were randomly assigned to the control group were not eligible to participate in the program during this period.
Across both cohorts, researchers measured two sets of outcomes: attendance rates (days absent and days present) and academic outcomes (grade point average, courses failed, and standardized test scores). These data were drawn from Chicago Public Schools student-level records.
Results and policy lessons
Researchers found that participation in the program increased student attendance and reduced the number of courses failed for students in grades 5-7, but had no effect for elementary school-aged children.
Impact on Absenteeism: Participants who began the program in grades 5-7 were absent an average of 3.4 fewer days than the control group average of 17 days (a 20.2 percent decrease)3 . There were no statistically significant effects on attendance for students in grades 1-4. The researchers hypothesize that there was no detectable effect for younger students because their attendance is more dependent on parents or adult caregivers, while older children have more independence.
Impact on Student Achievement: C&C had mixed results on academic outcomes for students in grades 5-7, and no effect for students in grades 1-4. Participating students in grades 5-7 failed 0.17 fewer courses than the control group base of 0.84 courses (a 20.2 percent decrease). However, average math standardized test scores among participating students in grades 5-7 were 0.19 standard deviations lower (the equivalent of a student in the 50th percentile moving to the 42nd percentile after program participation). The researchers hypothesize that this may be due to students in the treatment group being more likely to attend school the day of standardized testing. There was no effect on reading scores for students in grades 5-7 and no effect on standardized test scores for students in grades 1-4. There was no significant effect on students’ grade point average for either age group. While C&C increased attendance for middle school-aged children, the results suggest that increasing attendance alone may not be enough to improve academic outcomes for students with high absenteeism.
Cost Effectiveness: Researchers estimated the cost of C&C to be $1,700 per participant per year or approximately $500 per day of improved attendance. Though lower-touch programs, such as mailings to inform parents of absences, have been found to have smaller effects on attendance, they are substantially more cost-effective per day of improved attendance4 . Programs like Becoming a Man (BAM), an in-school cognitive behavioral therapy program for young men, have been shown to improve student engagement and simultaneously produce large improvements in graduation rates at a similar cost to C&C5 .