Both, educators and policy makers have expressed growing concerns over the low levels of math and science achievement among American students and the decline in the numbers of young people moving into science, technology, engineering, and math (STEM) careers (Committee on Prospering in the Global Economy of the 21st Century, 2007; National Science Board, 2012). The U.S. Bureau of Labor Statistics (2023) projects a nearly 11% increase in STEM occupations between 2022 and 2032, emphasizing the need for expanding the presence of highly-trained STEM workers. The proportion of women in STEM is especially troubling, with just about one-third of STEM workforce comprised of women (NCSES, 2023). While interest in expanding the numbers of young people moving into science and technology fields has grown, a relatively small proportion of STEM education evaluation has focused on the role that after-school programs can play to reinforce STEM learning and help engage young people in educational pathways leading to STEM careers. The FIRST Longitudinal Study, a unique semi-experimental study, fills this gap. FIRST® (For Inspiration and Recognition of Science and Technology) operates after-school robotics programs for young people ages 418. The mission of FIRST is to provide life-changing robotics programs that give young people the skills, confidence, and resilience to build a better world. Study Goals: This multi-method evaluation examines the impacts of FIRST on all participants' interests in STEM and their likelihood to pursue STEM in their academic and professional careers. Overarching evaluation goals are threefold: (1) Controlling for key demographics and income, what is the extent of differences in STEM attitudes and interest 10 years after program participation? (2) Are there differences in STEM outcome for key demographic, such as by gender, race/ethnicity, and income? (3) What differences in behavioral outcomes, such as STEM college course taking and majoring in STEM can be observed? Methods: Longitudinal survey data were collected in annual follow-up surveys of intervention group participants and a matched comparison group drawn over the 10-year evaluation period. Response rates have been consistently 72% responded in year 10. In-depth interview data with a diverse sample of 42 FIRST female participants were collected during the final year to better understand why the impact of FIRST is especially strong for women. Interview participants differed by race/ethnicity, family economic positions at program participation, and locations (urban, suburban, and rural). Expanding on the consistently higher impacts of FIRST for young women found in the analyses of the 10 annual post-program participation surveys, an interview study with 42 FIRST female alumni was added in the final year to fully understand these impacts. This addition was critical to include their voices reflecting on the especially strong program impact for women. Nationally representative and restricted data collected by the National Center for Education Statistics (NCES) create an additional comparison group through propensity score matching. (NCES data have not been analyzed yet. Results will be available at the SREE conference). Measures: Validated scales measure interest in STEM and STEM careers, involvement in STEM activities, STEM identity, and understanding STEM. For the behavioral outcomes, college course taking, college majors, and early career outcomes were analyzed. Control variables include gender, race/ethnicity, family income at baseline, parental support for STEM, and geography of participants schools (urban, suburban, rural). Qualitative interview data with a diverse sample of 42 FIRST female participants focused on post program participation topics such as STEM coursework in high school and college, the fields females were pursuing in their current careers and why, their experiences in FIRST, and how they believed FIRST influenced their post-FIRST academic and work experiences. Preliminary Results: Multivariate mixed-methods and logistic regression analyses of the 10th year survey data, controlling for baseline differences, reveal that FIRST college students reported significantly higher rates of STEM interests, attitudes, and college level behaviors than matched comparison students (Table 1). Significantly higher scale scores hold true among key demographic groups (low-income, racially/ethnically underrepresented groups in STEM, and by geographic location) and are strongest for female comparisons. Table 1: STEM Interest Scales at 120 months, Mixed Model. FIRST participants are also significantly more likely to take STEM classes overall, and engineering and computer science classes specifically than the comparison group. These higher rates link to significantly higher rates majoring in engineering or computer science (Table 2). Again, the results are stronger for female participants program participants. In contrast, the rates of college majors in the biological sciences and health professions are significantly higher for the matched comparison group. Preliminary analyses of early career data show that working in STEM is more important for FIRST alumni with incomes significantly higher for FIRST female alumni. Table 2: Declared Majors Year 4 in College. Preliminary analysis of the qualitative interview data underscore these impacts for women, who discussed how they got interested in joining FIRST and how their FIRST experience impacted their college majors. They also shared the challenges women experience when participating in FIRST and working in STEM fields, and how they persisted in STEM. This multi-method evaluation provides insights for longitudinal evaluation approaches and implication for the role of after-school STEM programs in the decision-making process regarding their STEM college and career choices, especially for young women. The evaluation also underscores the importance of including program participants' voices in evaluations.