Ideas to Implementation: How a research partnership can drive STEM equity and youth development in the city of Los Angeles

In J-PAL North America’s Ideas to Implementation blog series, our 2024 LEVER Evaluation Incubator partners share their experiences on the different steps in developing a randomized evaluation of their innovative programs. In part two of the series, J-PAL staff sat down with the Youth Development Department (YDD) of the City of Los Angeles and the University of Southern California (USC) to explore how they established their research collaboration for evaluating the Student Engagement, Exploration, and Development in STEM (SEEDS) program as part of the Evaluation Incubator.

To start off, please tell us about the program you are planning to evaluate and your evaluation goals.

The Student Engagement, Exploration, and Development in STEM (SEEDS) program is a culturally responsive, intergenerational mentorship initiative with a gaming-based learning component launched in 2022 by Dr. Darnell Cole (USC) and Dr. Christopher Newman of California State University, Fullerton (CSUF) in partnership with the Youth Development Department (YDD) of the City of Los Angeles. The program targets racially minoritized college students who participate in the mentorship program through an internship to mentor middle school students in a local college preparation program. College students are mentored by academic professionals from similar backgrounds (i.e., race, ethnicity, gender, socioeconomic class). Each SEEDS session utilizes an online game-based learning component, where students engage with STEM-based games to promote racial equity for Black, Indigenous, and People of Color (BIPOC) middle school and college students in Los Angeles. Our evaluation examines whether participation in the SEEDS mentorship program enhances STEM identity, community cultural wealth, degree attainment, and STEM career aspirations among college students. This asset-based program¹ provides culturally responsive mentorship to address the systemic barriers facing BIPOC middle school and college participants in a metropolitan urban setting. By addressing systemic barriers, SEEDS aims to advance educational, racial, and socio-economic equity in STEM pathways.

What factors influenced the University of Southern California’s (USC) decision to collaborate with the City of Los Angeles for this evaluation?

When we launched SEEDS, we saw a unique opportunity to partner with the then-new YDD of the City in several important areas to leverage resources, coordinate research, and strengthen research-practice-community collaborations. Notably, the City was crucial in leveraging local funding to incentivize college students to participate in SEEDS by providing paid internship opportunities for college students, facilitating partnerships with community-based organizations to hire, train, and compensate students who mentor Neighborhood Academic Initiative (NAI) middle school students, hiring additional college students who receive professional development, and providing administrative support to the SEEDS program. 

Who are some of the important stakeholders in this research partnership, and what considerations were taken in deciding roles and responsibilities?

The SEEDS project, led by Dr. Cole and Dr. Newman, follows a research design that is both systematic and meaningful. With the help of UCS researchers including Dr. Ting-Han Chang, Dr. Mabel E. Hernandez, Dr. Milie Majumder, and Dr. Tr’Vel Lyons, the team has built strong stakeholder relationships since the project's launch. In addition to YDD, key partners include local community-based organizations, the Neighborhood Academic Initiative (NAI) at USC, STEM professionals who mentor college students, and college students who, in turn, mentor middle school participants.

Shared goals between the research team, YDD, and community-based organizations have played a crucial role in the program’s success, allowing the research team to take the lead in implementation and research while receiving guidance and endorsement from these key partners. The City of Los Angeles has intentionally advocated for and allocated funds to support the SEEDS program by creating paid internship opportunities for the program’s college student participants, managed through its partnered community organizations. These organizations hire, train, and compensate students who mentor NAI middle school students. These organizations are dedicated to supporting youth development in the City of LA for those who come from underrepresented communities and often face barriers in their career development. The research team has thoughtfully engaged with these community organizations to advance the goals of the SEEDS program, including recruiting interns who meet the organizations' criteria and aligning with their mission to invest in underserved youth in LA. The research team continues to foster connections with STEM professionals nationwide to mentor college students and provide career guidance. 

How did the team decide on the key research questions, and how will these questions support program developments?

The research questions for this evaluation originated from a National Society of Black Physicists (NSBP) pre-conference workshop in 2017. The workshop explored strategies to increase African American participation in physics and astronomy and foster collaboration between high-producing and aspirational programs. Over the next few years, we refined our focus to the following research question: What is the impact of the SEEDS program, an intergenerational STEM mentorship program, on supporting minoritized college students’ STEM identity, college (and advanced) degree attainment, and economic mobility beyond college? The research team eventually connected with the City of Los Angeles officials through our school networks in 2021 and pitched the idea of SEEDS to continue supporting youth career development across the city.

By incorporating both quasi-experimental designs and randomized evaluation, study findings can help inform practices and policies that further support university-government and agency-community organization partnerships in providing culturally responsive mentorship, intergenerational mentorship, and professional development opportunities to BIPOC college students across the city. The findings could also help scale the program to other cities focused on youth development and career preparation, especially BIPOC, first-generation, and/or low-income students from urban city settings.

What are some of the key challenges involving navigating different priorities, timelines, and institutional structures you’ve encountered so far, and how have you addressed them?

One key challenge is aligning the SEEDS program’s timeline with various stakeholders, including the city, community organizations, NAI, the Los Angeles Unified School District (LAUSD), and participating colleges, each with different schedules. In addressing this challenge, the research team has been very mindful and communicative with all the stakeholders to find workable schedules that accommodate everyone and meet the SEEDS program’s implementation and research purposes. For YDD, the primary challenge has been identifying and leveraging funds to support the wages of the college student participants, since the city is undergoing some financial challenges as a whole. Nonetheless, YDD remains committed to advancing the program and leveraging any funds available to support the program participants in future cohorts of the program.

What are your hopes for the impact of this research once completed? How do you envision the findings being used to inform policy and program decisions in the City of Los Angeles?

YDD has conducted research in the areas of Positive Youth Development and Youth Program Evaluation. The implementation and evaluation of the SEEDS program have allowed the city to put parts of that research into practice. For example, YDD has established partnerships to develop tools, training, and other resources for government employees who mentor youth across city departments, which builds on the value of the mentoring approach in SEEDS.  Overall, lessons from SEEDS  implementation and evaluation will inform future collaborations with YDD and academic institutions to pilot and assess youth programs. By using data-driven insights and strengthening an evidence-building culture in Los Angeles, YDD aims to inform policy and program decisions, ensuring that resources are directed toward initiatives with measurable impact. Ultimately, this research will shape a more coordinated and effective approach to improve youth services and outcomes across Los Angeles.

¹ Yosso*, T. J. (2005). Whose culture has capital? A critical race theory discussion of community cultural wealth. Race ethnicity and education, 8(1), 69-91.