Image
J-PAL Europe launches IDEE to advance evidence generation and use in the French education system
Posted on:
Marc Gurgand, J-PAL affiliated professor and IDEE’s scientific coordinator, speaks at a panel during the IDEE conference.
Photo: Mathieu Alexandre
“Globally, children are learning less in their school years than we would have expected,” says Esther Duflo, director and co-founder of J-PAL. “The system is not treating students with the quality it should be,” and, in France, there is a “rapid and disturbing decline in students' mathematical skills.”
To help address these and other challenges, J-PAL Europe, in collaboration with the French Ministry of Education (Direction de l'Évaluation, de la Prospective et de la Performance) and leading French universities, launched the Innovation, Data, and Experiments in Education (IDEE) programme in 2022.
This effort is motivated in part by growing demand for the rigorous scientific evaluation of education policies and programmes in France. The French Ministry of Education, and its statistical department in particular, has shown a strong commitment for greater and more rigorous evaluation. Still, while many innovative programmes have been developed to respond to challenges in the French education landscape, few among them have been rigorously evaluated.
In turn, this shortage of scientific evaluation limits the use of evidence in policymaking and teaching practices, and therefore the identification of the most effective education programmes. Furthermore, without close partnerships between policymakers, researchers, and practitioners, evidence generation and use becomes more challenging for all parties.
In an effort to reduce these barriers, IDEE has an eight year mandate (2022-2030) to foster evidence generation and use in education in France, with funding from the French National Research Agency. The programme helps education researchers conduct rigorous impact evaluations, including by facilitating access to administrative data and providing research resources and measurement tools. Furthermore, IDEE is well positioned to create enduring relationships between government, researchers, and practitioners by providing opportunities to collaborate and strengthen capacities for evidence generation and use.
Bringing stakeholders together
In November 2022, J-PAL Europe and IDEE hosted a conference to bring together stakeholders—primarily policymakers, practitioners, and researchers—and increase understanding about how impact evaluations can be used in education settings. The conference aimed to raise awareness about the added value of evidence, and impact evaluations in particular, in education. It also facilitated the exchange of ideas and research priorities at the crossroads of policy, practice, and research.
At the conference, IDEE’s scientific coordinator and J-PAL affiliated professor Marc Gurgand (Paris School of Economics) outlined the motivations of IDEE, saying:
"The programme provides a structural framework that seeks to align actors, actions, points of view, and objectives in the service of research that is itself in the service of the public education system, and that cannot be conducted without actors from across the entire French education ecosystem."
This mission led to numerous productive conversations. Alongside keynote addresses from Duflo and Edouard Geffray (French Ministry of Education), multiple panels featuring education experts and practitioners allowed participants to converge on research and policy priorities. With topics ranging from research on math card games to teenage sleep patterns, participants engaged in multidisciplinary discussions where they touched not only on the process of evidence generation but also on how it can be further incorporated into education policy decisions.
The cognitive neuroscientist Stanislas Dehaene (Collège de France) also joined the event and anchored an evidence-sharing panel discussing research results across different domains. Panelists also shared some of the constraints of running randomized evaluations. For example, J-PAL affiliated professor Élise Huillery (Université Paris-Dauphine) dove into an evaluation she conducted aimed at reducing the high-school dropout rate in France via a socio-emotional skills programme. The evaluation, which took place over a period of five years, found that in-class discussions focused on developing skills like motivation, perseverance, and self-discipline had numerous positive impacts: students, and girls in particular, had better learning outcomes and reported improved moods and set higher goals.
In another notable panel, Lou Aisenberg, IDEE’s strategic development manager, along with researchers, policymakers, and practitioners, discussed how to best co-construct a research agenda that both builds on existing evidence and meets the needs of policymakers and practitioners. To achieve this goal, panelists noted the importance of creating space and time to align partners’ priorities and account for both methodological and on-the-ground constraints.
Featured evaluation: Increasing math skills in France through card games
In 2021 and 2022, the IDEE team provided methodological support to develop two evaluations of math card games. The first programme, called ‘Well played!,’ invited approximately 2,500 first grade students from more than 100 classes in France to play various math games during the summer holidays of 2021.
To measure the impact of this programme, the research team used national tests to assess students' performance in math. Students' confidence in math and their cooperation with each other was also measured using a specific test. The study results showed no effects on children’s learning, which could be due to the fact that it was conducted during the summer holidays.
In turn, this informed the development of a new math card game titled ‘Counting birds’, which was distributed in November 2021 to more than 65,000 students, in first grade classes throughout France. These cards, developed by the French Ministry of Education, can be broken down into several games that leverage different fundamental math skills. The aim of the programme was to build interest and encourage students to practice their math skills through playing games at home. National assessments were used to test students’ progress in mathematics, and their anxiety toward math was also measured using a separate survey at the end of the school year. The results of this intervention are forthcoming.
Looking forward
As IDEE enters its second year, the programme is building on the momentum from November’s conference with regular research seminars, methodological support for research teams running (or interested in running) randomized evaluations, and trainings supporting education policymakers and practitioners across France to better understand and use evidence in their decisions.
To raise awareness and further promote evidence-use in policymaking and teaching practices, IDEE is also set to launch an online class titled “Evidence-informed education: an introduction to impact evaluations."
This multi-pronged approach of direct and indirect engagement is laying the foundation for a stronger research infrastructure and an evidence-informed revitalization of the education ecosystem in France. As Duflo said, “There is no magic wand that takes scientific knowledge and applies it to the whole world.” In fact, collaborations between researchers and policymakers allow for an iterative research process that examines both successes and failures. In turn, the goal of improving educational outcomes comes closer to being met.
Interested in learning more? Visit IDEE’s website or contact Lou Aisenberg. You can also dive deeper into the topics discussed at the conference.
