How randomized evaluations build evidence to inform workforce program design, policy, and investment

Access to high-wage job opportunities in the United States is largely predicated on holding a four-year college degree. In the mid-20th century, most workers experienced rapidly growing wages in a period of shared prosperity, Katz explained. Since 1980, people with different levels of education have experienced widely divergent trends in earnings.

This growing divide means fewer paths to middle-class jobs, financial security, and prosperity for people without equal access to higher education or who don’t thrive in traditional postsecondary programs without further support, Katz said.

Using four-year degrees as an essential credential creates barriers to opportunity that disproportionately hurt Black and Latinx workers, Jones added. More than 70 percent of jobs that pay $40,000 or more require a degree. But, because of persistent and historic racial inequality that has excluded Black and Latinx Americans from economic and educational opportunities, three-quarters of Black workers and more than 80 percent of Latinx workers in the United States do not have four-year degrees.

“The bottom line is we have a credential in our country that is literally a systemic barrier to most of us earning our way into the middle class,” Jones said. “We’ve got to move to a skills-first approach, we’ve got to understand that there are multiple ways to come about those skills. The four-year credential is one way; it should not be the only way.”

Evidence of the effectiveness of sectoral employment programs is strong. A recent J-PAL North America review summarizing results from four studies of nine distinct sectoral employment programs found that such programs significantly increase earnings in the year participants completed their training and in subsequent years.

Moreover, these earnings gains are among the largest found in evaluations of US training programs. The gains are driven by getting people into higher-paying jobs that have higher earnings in the short term and strong upward trajectories over time.

The review also finds the most effective programs include a combination of several core elements: upfront screening for applicants on their basic skills and motivation, career readiness training, wraparound support services for participants, strong connection to employers in the target field, and occupational skills training for high-wage sectors that lead to industry-recognized certificates. These factors help explain why sectoral employment programs have stood out from other training programs with less comprehensive models that have not produced such impressive results.

Overall, the impact of sectoral employment programs is promising, and the evidence is compelling because it comes from a series of randomized evaluations, Katz explained.

Here are how randomized evaluations of the programs worked: Programs had a limited number of slots to fill and more applicants than they could serve. The slots were allocated through random assignment, as in a lottery, which ensured that people who received access to the program were not, on average, systematically different from those who did not. Therefore, subsequent differences in employment, earnings, credentialing, and other outcomes can be attributed to participation in the training programs and not other factors

Questions remain despite the strength and promise of evidence on sectoral employment programs. These questions include how such programs affect different groups of workers (for example, people without high school diplomas), what range of industries and salaries they can successfully prepare workers for, and how they can be sustainably financed.

The WorkRise-funded evaluation of the Pursuit Fellowship program seeks to contribute valuable evidence on some of these questions. But what motivated Pursuit’s desire to evaluate? Pursuit already has promising nonexperimental results showing that participants’ average annual income climbs from $18,000 before joining the program to $85,000 after graduating—so why participate in a randomized evaluation?

“Outcomes are fundamental,” Pursuit cofounder Jukay Hsu said. “We all care about creating great outcomes for people, for families, for employers, for communities. Without evidence and outcomes, there is no point in doing the work daily, and I think we should be really rooted to that.”

Hsu noted that a randomized evaluation will also enable Pursuit to share lessons with other programs, policymakers, and investors and funders, facilitating much-needed learning.

“We want to understand the counterfactual, we want to understand how this influences policy,” Hsu said. “And it’s not just to make our work better at Pursuit. Hopefully, we can work with others to take part of our model or adopt others so we can support development and growth in the field overall.”

All panelists echoed this last point, agreeing that addressing income inequality and barriers to job opportunities will require systems change, a playbook of evidence-based strategies, and engagement with many different actors in the labor market. “The work really is about building an ecosystem,” Jones said.

“We need multiple options that work,” he added. “We definitely need 1,000 flowers to bloom to reach the talent and maximize the opportunity that we have here.”

Building an evidence-based playbook of strategies to equitably increase economic security is imperative to address multiple challenges and barriers to opportunity facing workers. Many innovative ideas and program models exist; achieving greater worker prosperity will require testing those models, as illustrated by J-PAL North America’s evaluation of the Pursuit Fellowship, and sharing those lessons broadly.