Structured Study Time, Self-Efficacy, and Tutoring
- Higher education and universities
- Students
- Dropout and graduation
- Student learning
- Coaching and mentoring
- Commitment devices
- Digital and mobile
- Information
- Nudges and reminders
- Student motivation
Massive online courses have the potential to make quality higher education accessible to a much larger public, but they have been plagued by low retention rates. In this study, researchers evaluated scalable methods to improve student retention and performance in an online course, with the goal of improving meaningful access to this resource.
Policy issue
Massive Open Online Courses (MOOCs) have expanded in number and popularity but have exhibited relatively low retention and completion rates. MOOCs are potentially at a disadvantage compared to in-person instruction in several ways. First, standard education provides a structured study environment (classes, assignments, etc.) which most students may need to successfully complete a course. Second, standard in-person education provides one-on-one interactions with an instructor who may be able to spot an individual’s problems or questions and address them before the student gets completely lost. Third, students can come from all over the world and those students who are far away or do not share the same language may have lower confidence about their ability to succeed. However, there is little evidence on scalable interventions designed to address these barriers and whether they can lead to increased retention, completion, and learning. Can blocking out regular study time to interact with online courseware yield better retention and performance? Could extra tutoring from a teaching assistant lead to learning gains? Could encouraging messages course boost marginalized groups’ performance?
Context of the evaluation
Researchers ran their evaluation as part of the edX course “The Challenges of Global Poverty” in spring 2014. This is a course that the researchers ran once before in 2013 and provides an introduction to the main topics in global poverty as conceptualized by leading economists and political scientists. Anyone can access the course for free online. The language of the lectures is English and the course is run over 13 weeks. Participants who complete the course can obtain a verified certificate for a fee. In 2014, nearly 20,000 students registered for the course. Of these, 24 percent were active, 6 percent achieved a grade of at least 50 percent, and 6 percent were active on forums.
Details of the intervention
Researchers conducted a series of randomized evaluations to measure the impact of three sets of interventions on online course retention, interaction, completion, and exam grades.
Structured study time: A randomly chosen subset of students had the option to commit to a regular study time. Students who opted in were required to record the time or times that they planned to dedicate to the course each week. Researchers also tested the impact of various enforcement mechanisms to help students commit to studying. One enforcement mechanism was a message sent to a random subset of students stating that the course staff can monitor whether students were using their designated time to study by looking at the timestamps of when students log in and log out. The second was a set of email reminders sent either one-third of the way through the course, two-thirds of the way through the course, or both. These reminders encouraged students to stick to their committed study time and indicated how closely they had been adhering to that time.
Self-efficacy messages: Self-efficacy refers to someone’s belief that they have the capacity to act in ways that achieve specific performance goals. Sharing information with students as to who performs well in online courses is thought to boost the effort of students and their expectations of what they can achieve. The course entrance survey included self-efficacy messages that provided factual information on who did well in the course the previous year. Students were randomly allocated to see either no message or one of three messages: (1) a generic message (2) a message related to females performing well in the course, (3) a message related to non-native English speakers performing well in the course.
Tutoring: All students that enrolled in the course were offered the opportunity to enter a lottery for tutoring. Of those that signed up, 500 were randomly selected to receive tutoring with a group of 20 other students. Tutoring services consisted of weekly online group review sessions, availability for individual questions over email (on assignments or on lectures) on a weekly basis, and a final exam group review session. The tutor played the role that teaching assistants play in in-person education. Researchers monitored the level of engagement between tutors and tutees, as well as the effect of having access to a tutor on eventual performance.
Results and policy lessons
Study ongoing; results forthcoming.