Artificial intelligence to strengthen high school students’ writing skills

In Brazil, public school students face significant challenges in developing writing skills, partly because teachers often don’t have time to provide individualized assistance to students. There are wide achievement gaps between public and private school students in Brazil, and limited literacy can prevent students from succeeding later in their careers. In 2019, the state of Espírito Santo piloted an artificial intelligence (AI) platform called Letrus to give public school students rapid feedback on their writing as they prepared for the national university admissions exam. A randomized evaluation found that the intervention led to improvements in students' writing abilities and increased their scores on the entrance exam, helping to narrow the achievement gap between public and private school students. Drawing on these findings, the state rolled out the platform to high school seniors statewide, and Letrus is now pursuing partnerships with other state governments across the country. 

The Problem

In Brazil, public school students face significant challenges in developing writing skills essential for success in the national university entrance exam, ENEM.

While Brazil has made progress in school attendance and literacy rates in the last two decades, functional literacy skills remain low.¹ On the Programme for International Student Assessment (PISA), a worldwide study of scholastic performance, the average Brazilian 15-year-old scored 413 points on reading in 2018, whereas the average 15-year-old from an OECD country scored 487 points; only two percent of Brazilian students received top marks.² One barrier to improving language skills is that teachers, who are busy with tasks like preparing lessons and grading, may not have the time to provide students with individualized assistance like in-depth writing feedback. Additionally, students in public schools in Brazil often have less access to tailored resources and learning support than their private school counterparts.

The Brazilian national university entrance exam (Exame Nacional do Ensino Médio, or ENEM), which serves as a gateway to higher education in Brazil, emphasizes reading and writing skills. The gap in scores between public and private school students is significantly larger on the essay section than on other test sections. Disparities in both writing proficiency and university admissions can affect students’ future educational and career prospects. 

As students prepared for ENEM in the 2010s, the state government of Espírito Santo recognized the lack of tailored writing support for students and sought an innovative approach to address their specific needs.

The Research

Letrus, a Brazilian education technology company, developed an AI-based platform aimed at improving high school students' writing skills by providing them with personalized feedback on their essays. The platform is designed primarily for public school students in their senior year who are preparing for ENEM. 

In 2019, the Espírito Santo State Education Department (Sedu) and Letrus partnered with researchers—J-PAL affiliated professor Bruno Ferman (São Paulo School of Economics) together with Flavio Riva and Lycia Silva e Lima—to assess the platform's effectiveness in strengthening the writing skills needed for university admissions, with the goal of addressing the educational disparities between public and private school students. The opportunity to conduct a rigorous test of the platform’s impact on students’ writing skills helped convince the government to pursue the pilot; policymakers sought to ensure that limited public funding for education was being invested in a high-impact program.

The evaluation was conducted in 178 public schools, with approximately 12,000 students in 110 schools receiving access to the Letrus platform. The study tested two versions of this intervention: one providing only AI-driven feedback and another enhancing AI with human feedback. Students in both groups wrote five practice ENEM essays on the Letrus platform and received instant AI-generated feedback on style and syntax. In the “enhanced” version, students also received feedback from human scorers three days later. Students in the comparison group, comprising 68 schools, did not have access to the Letrus platform.

Students in both treatment groups wrote more practice essays, received more and higher-quality feedback, and discussed more essays individually with their teachers than students in the comparison group. Both interventions increased students’ entrance exam scores by about 0.09 standard deviations—equivalent to 17 points out of a possible 1,000 points—closing about nine percent of the public-private school gap in essay scores. The similarity of effects between the two interventions suggested that additional feedback from human scorers did not improve students’ essay scores more than the AI feedback alone. Both versions of the platform increased the number of training essays that students personally discussed with teachers, suggesting that the platform freed up teachers’ time grading essays to give more individualized assistance to students. 

This evaluation received generous support from Google.org through J-PAL’s Post-Primary Education Initiative. For more details on the research findings, see the evaluation summary.

From Research to Action

Based on these findings, the state of Espírito Santo formalized its partnership with Letrus and expanded access to the platform to high schools statewide. In 2022, the Espírito Santo State Education Department announced that the initiative would be scaled to reach all senior year students in public schools across the state. In announcing this decision, the Department cited the platform’s proven impact on student test scores. The Department chose to scale the AI-only model due both to its ability to achieve similar impacts at a lower cost and its prompt feedback mechanism, which was seen as a key factor in engaging students. 

Despite challenges posed by the pandemic, when students lost access to school infrastructure, the Letrus platform has since been widely adopted. Since 2020, more than 100,000 high school seniors have been enrolled in the platform, including over 28,000 students in 2024. In 2022, an average of 50 percent of students across the state submitted essays through the platform, rising to 60 percent in 2023. The program is now included in the state’s budget and fully integrated into the curriculum for high school seniors, demonstrating the state’s commitment to enhancing educational outcomes through innovative technology.

Letrus is also in the process of developing partnerships with other Brazilian states. It cites results from the evaluation when approaching new state governments and their education departments to propose partnerships. Through support from foundations, the states of Mato Grosso do Sul and Alagoas adopted the Letrus platform in some schools in 2021, followed by Paraíba and Pará in 2022. In 2024, Letrus secured direct partnerships with the states of Mato Grosso and Goiás.

Letrus has secured funding from USAID’s Development Innovation Ventures to develop a new randomized evaluation to study longer-term outcomes such as university enrollment and employment. The implementation is expected to run from 2024 to 2026 and include students from all public high school grades, expanding beyond the 2019 study's focus on seniors. 100,000 students are anticipated to receive the intervention as part of the study. The partnership with USAID will take place in the state of Ceará.

"The implementation of a writing platform for all students enrolled in their senior year of high school in our public education network reaffirms the commitment of the Government of the State of Espírito Santo, through its State Department of Education—Sedu—to continuously invest in innovative actions that positively impact the learning and future of these young people. Since 2019, this initiative has already benefited more than 60,000 students, proving to be a strong ally in promoting the development of writing and reading skills of the network's students."

— Statement from Sedu

References

Ferman, Bruno, Lycia Lima, and Flávio Rima. “Artificial Intelligence, Teacher Tasks and Individualized Pedagogy.” Working Paper, March 2021