Using rigorous impact evaluation to help people find jobs

Addressing unemployment is a persistent challenge for social policy in France. To help job seekers find employment, the government created the French Public Employment Service (known as Pôle emploi) in 2008. From its creation, the agency established an evaluation committee to ensure use of evidence in program design. 

Evidence from impact evaluations, including randomized evaluations conducted by J-PAL affiliated researchers, has informed Pôle emploi’s policy and program decisions. These studies have rigorously measured the effectiveness of public and private counseling, group counseling, and career counseling for young job seekers, among other programs. Agency staff have also participated in J-PAL trainings, strengthening their capacity to conduct impact evaluations. Applying research results and generating new evidence has helped Pôle emploi solidify a strong culture of evidence use. 

The Problem

France has seen high levels of unemployment over the past decades, including high youth and long-term unemployment.

In recent decades, France has experienced steady levels of unemployment above the OECD average, including long-term unemployment, which currently affects almost 40 percent of French job seekers.¹² The unemployment rate of young people aged 15 to 24 is more than three times the national average.³ 

In addition to the negative effects of unemployment on the individual, such as declines in income, deterioration of mental health, and lower future wages, short- and long-term unemployment can have negative consequences at the societal level, potentially leading to productivity losses and reduced social cohesion. Young people often face additional barriers related to unemployment, such as difficulties finding a first job or securing long-term, stable employment. 

The French government established Pôle emploi, a public employment service provider, in 2008 to simplify the job search by serving as a single, centralized point of contact for job seekers when they register as unemployed.⁴ Job seekers now have an initial interview at Pôle emploi, after which they are appointed an advisor responsible for providing them with long-term support. Services provided by the agency include job search support, personalized job counseling, training, recruitment assistance for firms, and more. 

The Policy Solution

From its inception, Pôle emploi has incorporated a culture of evidence use into its decision-making process. 

Pôle emploi seeks to ensure that the services it offers are both innovative and evidence-based. To assess the efficacy of its programs, the agency established a strategy and evaluation committee (Comité Stratégique et d'Évaluation), which includes Pôle emploi representatives and external evaluation experts. The committee guides the institution’s evaluation strategy, aiming to better understand and communicate evaluation results.

The committee strives to ensure that a new evaluation will help understand not only the impact of a specific program but also any underlying mechanisms that can explain results across studies. The committee proposes evaluations to the agency’s Board of Directors for approval.

When an evaluation is completed, the committee shares the results with the Board and provides recommendations for any relevant changes to Pôle emploi’s services. This process provides a clear pathway for applying lessons learned from evaluations to help more people successfully find employment.

“At Pôle emploi, evaluation is at the very heart of political choices. From the beginning, it was extremely important for us to ensure that evaluation is part of the creation of new labor market policies, and that there is no strategy formulation without the incorporation of results from evaluations.”

— Patricia Ferrand, Deputy Chair of Pôle emploi’s Board of Directors and Chair of the Strategy and Evaluation Committee of Pôle emploi.

From Research to Action

Evidence from randomized evaluations has informed Pôle emploi’s decision-making processes, helping the agency improve its programs and driving the direction of its future research.

The creation of a committee charged with policy evaluation reflects Pôle emploi’s commitment to evidence-based decision-making, which has led to a cycle of innovation, experimentation, and learning within the institution. 

Since 2008, J-PAL affiliates have conducted several randomized evaluations in partnership with Pôle emploi, as well as the missions locales (local missions)–a separate entity that provides support to youth under 25. These evaluations have rigorously measured the effectiveness of their programs, including public and private counseling, group counseling, and career counseling for young jobseekers. These evaluations came after a series of observational studies during which J-PAL affiliated researchers Bruno Crépon and Marc Gurgand introduced Pôle emploi staff to randomized evaluations—at a time when this type of public policy evaluation was very novel in France.

One of the earliest examples of this collaboration was a study by J-PAL affiliates Luc Behaghel (Paris School of Economics), Bruno Crépon (ENSAE), and Marc Gurgand (Paris School of Economics). The researchers carried out a randomized evaluation of two programs involving nearly 44,000 job seekers at risk of long-term unemployment to measure the impact of reinforced counseling offered by private contractors and by the state. Intensive counseling helped participants find work sooner than the standard low-intensity counseling program, and the counseling was more effective when provided by a public agency than by private contractors. 

Results from this study, combined with other supporting evidence from across Europe, prompted discussions that eventually led to a shift in the agency’s outsourcing strategy: towards providing in-house intensive counseling for job seekers most at risk of long-term unemployment, while contracting service providers to work with more independent job seekers. This allowed Pôle emploi to focus staff efforts on supporting people who faced the greatest obstacles to employment.

Interest in this early study led to broader questions, such as potential issues of displacement effects, where improvements in employment outcomes for counseling participants could come at the expense of job seekers who do not receive counseling. In response, J-PAL affiliates Bruno Crépon, Esther Duflo (Massachusetts Institute of Technology), Marc Gurgand, Roland Rathelot (ENSAE), and Philippe Zamora (French Ministry of Labour) measured the impact of a career counseling program for young, college-educated job seekers to assess whether counseling created a displacement effect. 

Counseling helped participating individuals obtain stable employment in the short term but had no impact on their employment outcomes in the long term. The initial positive impact did create a displacement effect, particularly in job markets where there were few jobs available.

Pôle emploi has over time shifted its strategy from focusing almost exclusively on improving job seekers’ employment outcomes to working with private companies to identify and respond to the key recruitment challenges they face. For example, in 2015, Pôle emploi began offering free recruitment services for firms. 

In another recent evaluation, researchers Bruno Crépon, Yann Algan (Sciences Po), and Dylan Glover (INSEAD) partnered with Pôle emploi to assess how government-provided recruiting services could reduce firms’ recruitment costs and improve employment outcomes. The recruiting services helped firms reduce their hiring costs and led them to increase their job vacancy postings and hire more workers, compared to firms that were not offered the services. (A separate evaluation launched in 2009 by Luc Behaghel, Bruno Crépon, and Thomas Le Barbanchon (Bocconi University) examined the impact of anonymizing résumés reviewed by employers, but found that this intervention  did not have the desired effect in reducing hiring discrimination–see more details here.)

Ongoing research is building on results from previous studies and exploring a new set of policies targeting private companies. This includes, for example, using artificial intelligence to improve job recommendations to job seekers and measuring the impact of remote vocational training programs. Another ongoing evaluation concerns an intervention that makes job recommendations based on job seekers’ relative preferences regarding future wage, location, occupation, etc., as opposed to the current practice, which gives a fixed weight to each criterion. Researchers are also evaluating interventions that help job seekers form realistic expectations and reframe their searches in response to relevant information.

Often, a major part of the costs of running an impact evaluation is data collection. One way for public administration bodies to reduce the cost burden is by using existing administrative data. Many of these studies with Pôle emploi leverage administrative data and are implemented by caseworkers as part of their regular interactions with job seekers, making the evaluations low-cost. 

The long-standing research collaboration between Pôle emploi and J-PAL affiliates means that evaluation design often involves active participation of agency staff, some of whom have participated in J-PAL impact evaluation trainings. As a result of this collaboration, Pôle emploi staff have developed a strong understanding of randomized evaluations and capacity to conduct evaluations independently, helping to ensure that their programs are increasingly effective in helping job seekers find stable, long-term employment.