How Malleable are Non-Cognitive Skills? Measuring the Impact of Increasing Grit in Turkey
- Children
- Primary schools
- Teachers
- Urban population
- Student learning
- Training
- Soft skills
Although non-cognitive skills, including “grit,” may be important for both academic achievement and labor market success, little is known about how malleable these skills are. Researchers worked with public elementary schools in Istanbul to measure the effects of a “grit curriculum” on fourth-grade students’ perseverance, competitiveness, and test scores. Students who were exposed to the grit curriculum in 2013 earned higher scores on math exams, and the program also reduced the gender gap in competitiveness by encouraging girls who were likely to succeed to compete. When the program was delivered to a second set of Istanbul elementary schools in 2015, researchers found that students who were exposed to the grit curriculum increased their perseverance, but were also less altruistic towards unsuccessful peers.
Problema de política pública
Evidence suggests that non-cognitive skills such as patience, self-control, and conscientiousness affect various individual outcomes, including educational attainment, criminal activity and health. However, less is known about the role of grit, generally defined as perseverance in a productive task, especially as it relates to goal-setting, effort, and altruism. Grit may influence a wide range of beliefs, decisions, and outcomes, including entrepreneurial success and college dropout rates.
Moreover, a growing body of evidence suggests that non-cognitive skills like grit are malleable, primarily during childhood, and that they can be developed through educational interventions. Teachers may therefore contribute to improving children’s grit by highlighting the importance of goal-setting and perseverance in achieving success. They may also help students realize that their skills can be improved through effort, leading to increased perseverance, competitiveness, and achievement.However, emphasizing the role of effort in achieving success may also have unintended effects on altruistic behaviors. Individuals who believe that hard work can lead to success may be more likely to attribute personal responsibility to a bad outcome and therefore be less sympathetic towards those who do not succeed.
Given the pivotal role of non-cognitive skills for academic achievement and labor market success, can training teachers to encourage perseverance and goal-setting increase grit, leading to better educational outcomes among students? Does promoting effort-oriented achievement affect students’ altruistic behavior?
Contexto de la evaluación
Primary school participation and efficiency, as well as students’ performance, are main concerns regarding education improvement in Turkey.
For up to five “official extra-curricular” hours each week, public school teachers in Turkey are encouraged to participate in social programs offered by the government, private sector, international organizations, and NGOs in a variety of topics. The program studied in this evaluation was overseen by the Turkish Ministry of Education as part of this national initiative and was offered by a major international bank. The program mainly reached students from lower socio-economic backgrounds, since most middle-class families in Turkey tend to send their children to private schools.
Detalles de la intervención
Researchers worked with public elementary schools in Istanbul to evaluate the impact of an innovative education intervention on students’ grit, educational outcomes, and altruism. Each school where at least one fourth-grade teacher stated his or her willingness to participate in the program was randomly allocated to either the treatment or comparison group. Teachers in treatment schools were trained to deliver a grit curriculum during “official extra-curricular” hours. The curriculum included animated videos, case studies, and classroom activities highlighting:
- the ability to develop new skills over time,
- the role of effort in enhancing skills and achieving goals,
- the importance of persevering in response to negative performance feedback, and
- the importance of goal setting.
During the training, teachers were also encouraged to emphasize the role of effort in typical classroom tasks, extending the themes from the grit curriculum into everyday classroom practices. Most teachers delivered the grit curriculum in about twelve weeks, spending about two hours each week on the content.
Researchers evaluated the program in two separate studies with different samples:
- Sample A consisted of 36 elementary schools, reaching about 1,700 fourth-grade students. 15 schools were randomly assigned to receive the grit treatment and the remaining 21 schools served as the comparison group. Treatment schools in Sample A received the grit curriculum in 2013.
- Sample B included 16 elementary schools, reaching about 1,300 fourth-grade students. Eight schools were randomly assigned to receive the grit treatment and the remaining eight schools served as the comparison group. Treatment schools in Sample B received the grit curriculum in 2015.
Around four months after the curriculum was delivered, researchers visited schools to measure students’ beliefs, grades, test scores. They also developed innovative mathematical tasks to measure competitiveness, effort, and altruism.
To measure competitiveness, researchers introduced an addition task that was conducted in three rounds. In the first round, students were rewarded with one token for each correct answer they gave. (Tokens could be exchanged for gifts at the end of the researchers’ visit.) In subsequent rounds, students decided how they would like to be rewarded for their performance: they could choose to be paid one token for each correct answer as before, or to compete against a randomly-selected student from a different school. If a student competed and answered more questions correctly than their opponent, the student received three tokens per correct answer; otherwise, the student received zero. After the second round, students received feedback on how many questions they answered correctly, as well as their performance relative to others, before deciding how they would be rewarded in the third round.
To measure effort, researchers designed a new task, which was composed of five rounds and consisted of looking at a grid of different numbers to find pairs that added up to 100 within 1.5 minutes. Before each round, students could choose between a difficult task that paid a large reward or an easy task that paid a small reward. When the game was done, one of the rounds was selected by chance and the students got rewards based on their performance in that round. Researchers conducted the same exercise the following week. At the end of the first visit, students were given practice materials and were asked to choose which task (easy or difficult) they wanted to perform the following week.
Finally, researchers measured altruism (only in Sample B). After the second round of the effort task, all students were asked to donate a fraction of their earnings to anonymous students who did not have any tokens. A randomly selected half of the students in the treatment and comparison groups were informed that these other students did not have tokens because they never had the chance to participate in the game (“unlucky” students). The remaining half of students in both groups were informed that the other students had failed the game despite having the chance to practice (“unsuccessful” students). Researchers measured students’ willingness to give in these two contexts to identify whether the effort-oriented curriculum changed students’ sympathy towards others. In both samples, researchers also collected students’ grades and test scores up to 2.5 years after the curriculum was implemented.
Resultados y lecciones de la política pública
Overall, students in schools with the grit curriculum were more likely to engage in challenging tasks, persevere after negative feedback, set goals and engage in skill-building activities, as well as score better on math tests. The grit curriculum helped to close gender gaps in the willingness to compete both before and after receiving feedback. However, the curriculum diminished students’ sympathy towards others who were unsuccessful.
Increased grit: Students in schools with the grit curriculum were more likely to opt for a difficult, high-reward task when offered the choice against an easier, low-reward alternative. They were also more likely to re-attempt the difficult task after receiving negative performance feedback. In addition, these students were more likely to succeed at the challenging task during the second visit, suggesting that they exerted more effort to improve their performance than students in comparison schools, likely by using practice materials. Researchers hypothesize that the grit curriculum increased students’ beliefs that abilities were malleable and could be improved through goal-setting and hard work, leading to higher test scores.
Willingness to compete: After participating in the grit curriculum, both boys and girls were more willing to compete in the first round of the addition task, suggesting that competitiveness is malleable. Girls in treatment schools were 14 percentage points (41 percent) more likely to compete in the first round relative to girls who did not participate in the grit curriculum, of whom 34 percent competed. Boys in treatment schools were 8.6 percentage points (20 percent) more likely to compete relative to boys who did not participate, of whom 42 percent competed. Researchers posit that students were more likely to compete because the grit curriculum made them more optimistic about their own performance.
Closing the gender gap in willingness to compete: Regardless of whether they received positive or negative feedback, girls in the intervention schools were about 15 percentage points (37.5 percent) more likely to compete in the second round of the addition task than girls in comparison schools, of whom 40 percent competed. Moreover, girls who were more likely to succeed were more likely to compete. The program had no impact on boys’ willingness to compete after receiving feedback, meaning that the grit curriculum eliminated the gender gap in competitiveness.
Grades and test scores: The grit curriculum had large impacts on students’ test scores. Immediately after the intervention, researchers estimated an increase of 0.31 standard deviations on math scores and 0.13 standard deviations on verbal scores for Sample B. Results also suggest that the positive impacts on math scores persisted for at least 2.5 years in Sample A and 1.5 years in Sample B.
Altruistic behavior: When asked to donate to “unlucky” students, there was no difference in giving between students in intervention schools and those in the comparison group. However, curriculum students were 11 percentage points less likely to donate when asked to give to an “unsuccessful” student, compared to 36 percent of students in the comparison group (a 30 percent decline). Thus, while the intervention did not have an impact on the overall levels of generosity amongst students, it led to less sympathy towards those who had been given the chance to succeed but nevertheless failed. Researchers hypothesize that students’ shifting beliefs about the role of hard work versus innate ability drove this change in altruistic behavior. Students who participated in the grit curriculum were more likely to attribute failure to insufficient effort, rather than low ability, and therefore felt less sympathy towards those who had been unsuccessful.