Policymakers and researchers collaborate to improve education quality in Puerto Rico

Puerto Rico faces deeply rooted and complex education challenges. We formed a partnership between the Puerto Rico Department of Education (PRDE) and academic researchers to study these challenges and promote decision-making based on data, research, and evidence. Through a collaboration built on trust and a common agenda, we aim to transform the Puerto Rico education system to improve youth academic achievement.

An education system facing historical and deeply rooted challenges

Puerto Rico’s school system faces longstanding challenges that have resulted in historically poor student academic performance. In 2017, for example, National Assessment of Education Progress (NAEP) test scores showed that 85 percent of fourth graders in Puerto Rico did not demonstrate basic proficiency in mathematics, as compared to 21 percent of their peers in the nation.

These challenges were further exacerbated by the fiscal crisis and the devastation inflicted by Hurricane Maria in 2017. Owing more than $70 billion to creditors, Puerto Rico had filed for bankruptcy-like protection a few months before Maria hit. And when the hurricane made landfall in September 2017, it inflicted an estimated $90 billion worth of damage to the island.

Prior to 2017, student enrollment had been steadily decreasing due to the decade-long economic recession. Following Maria, this situation worsened as more families relocated to the US mainland. The Puerto Rico government now estimates that their school system has lost more than 125,000 students since 2014 and now serves about 307,000 students.

Addressing the education crisis

The PRDE is taking a multi-prong approach to address these challenges. In the summer of 2018, the PRDE closed over 250 schools to allow finite funds to continue to provide a full set of resources—including a full teaching staff—at every open school.

Six pillars of education reform were also introduced, including:

1. A new education bill, which passed in March 2018;
2. The creation of seven regional offices with a Superintendent leading each;
3. A clear definition of the resources that each school needs;
4. Programming priorities for academic and student services;
5. A transparent, restructured budget; and
6. A new pay scale and overall salary increase for principals and teachers.

Critically, the PRDE also turned towards an evidence-based approach to tackle important education policy questions. And an emerging partnership between the PRDE, J-PAL North America, and academic researchers served as the catalyst behind this new approach.

Foundations of a new partnership

This latest partnership stems from previous relationships with various segments of the Puerto Rican government developed over the past decade by J-PAL affiliated researchers. In 2013, Marco González Navarro (University of California, Berkeley), and I (Gustavo) first connected with the Puerto Rico Administration for Socioeconomic Development of the Family (ADSEF). This initial collaboration with ADSEF focused on identifying opportunities to evaluate programs to support individuals with low-income in finding and maintaining employment.

In November 2016, after a new governor was elected, researchers and staff from J-PAL’s Global, North America, and Latin America & the Caribbean offices visited Puerto Rico for an extensive series of meetings with the new administration.

During these meetings, the education sector was identified as a top priority in the government's agenda, with poor student outcomes and ineffective bureaucratic and management structures highlighted as major concerns.

Government leadership expressed interest in J-PAL’s evidence-based approach, and members of the administration signaled their openness to using data and evidence, as well as input and collaboration from external partners, as tools for overhauling the education system.

This initial step of listening to the new administration and understanding its priorities helped open the door to broader collaboration.

As an outcome of this engagement, I (Nicolás) was hired by J-PAL North America to be embedded as the Chief Evaluation Officer at the PRDE, where I was tasked with building evaluation capacity within the department. What was initially envisaged as a six-month position later turned into a two-year engagement.

Embedding this Chief Evaluation Officer position proved important for several reasons:

1. First, it signaled to the PRDE that we were genuinely interested in and committed to understanding the agency’s needs and priorities.
2. The position also played a key liaison function between the different parties involved in the partnership—providing timely and precise communication that could be adapted to the distinct agendas and schedules of both policymakers and researchers.
3. It was also critical to identifying opportunities to build evaluation into PRDE projects and initiatives.
4. And, finally, it set the foundation for building PRDE staff capacity in evidence-based decision making by providing guidance on how to incorporate randomized evaluations as a component of program design.

A promising research agenda

Embedding me (Nicolás) within the PRDE paved the way for better understanding the agency’s strategic goals—allowing for the development of a joint research agenda aimed at reducing inequality in student academic achievement. The research team eventually expanded to include researchers Daniela Scur (Cornell University) and Orlando Sotomayor (University of Puerto Rico Mayagüez).

Together, we identified high-priority projects where rigorous evaluation would be most useful in answering key policy questions for the PRDE. One priority is to understand the implications of recent school closures in order to build on any positive effects and identify policies to reduce potentially negative outcomes.

Another top priority is to evaluate management-level interventions seeking to improve student learning.

Management-level interventions have been shown to lead to improvements in school quality and students’ academic achievement, but the adoption of these practices in the public sector is often not sustained in the long run. The PRDE and its academic partners became interested in understanding how successful management-level interventions can be promoted sustainably and at scale.

Together with the PRDE, we designed an evaluation of EDUGESPRO—a large-scale principal training program seeking to improve managerial practices and student achievement. The program, which commenced in June 2019, will be implemented among all school principals in the island’s public education system over a three-year period.

Catalyzed by early learnings and successes, we began exploring opportunities to sustainably fund the embedded evaluation position and the growing research agenda. We were thrilled to receive an Institutional Challenge Grant Award from the William T. Grant Foundation and the Spencer Foundation in April 2019 to continue the partnership’s work.

Coupled with funding from the J-PAL State and Local Innovation Initiative and Social Science and Humanities Research Council grants from the Government of Canada, the partnership will begin implementing our research agenda in earnest in June 2019. And, critically, the Chief Evaluation Officer position at the PRDE will be sustained for another three years.

Lessons from the partnership

Among the partnership’s greatest successes are the institutional changes it forged. At the PRDE, there is now a commitment to develop and formalize internal processes to increase the use of research, and the department looks forward to creating an internal team that can lead this evaluation work in the future.

At my (Gustavo’s) institution, the University of Toronto, the Department of Economics was interested in facilitating partnership engagements between faculty members and public agencies. The academic partnership with the PRDE served as an impetus.

Today, the University is developing a policy research lab that formalizes a framework, based on the PRDE partnership, to engage faculty and public agencies and facilitate research-practice partnerships.

Reflecting back, we believe the partnership’s success hinged on: setting clear expectations; leveraging each partner’s unique expertise; and crafting a common agenda grounded on problems of practice.

This collaboration between policymakers and researchers was also the result of carefully cultivated mutual understanding—investments in building strong relationships, developing trust, identifying mutual interests, and finding a common ground led us to where we are today.

Most importantly, this partnership framework ensures that the research questions being addressed are pertinent to policymakers and, ultimately, to the people of Puerto Rico. Through this approach, we aim to ensure that evidence informs decisions and policymaking, and that those informed decisions contribute to improved education quality in Puerto Rico.