This study explored STEM program quality prior to and during the COVID-19 pandemic, including differences based on learning setting. Program quality data was drawn from a national database consisting of 1259 program quality observations conducted between 2013 and 2021. Using the Dimensions of Success (DoS), an observation tool focused on informal STEM learning program quality, we analyzed 12 dimensions of quality obtained from 1212 observations gathered prior to the COVID-19 pandemic, a subsample of the national dataset (n = 616) including only states who submitted DoS observations before and during the pandemic, and 47 observations gathered throughout the pandemic. National trends in program strengths and challenges were replicated in the pandemic sample. Compared to observations conducted prior to the pandemic, the pandemic sample demonstrated significantly weaker evidence of quality in the dimension of Engagement with STEM, which considers the extent to which activities promote physical and cognitive learning. Findings revealed no significant differences across the 12 dimensions between in-person and virtual observations conducted during the pandemic. Common frameworks to support evidence-based planning relevant to dimensions that are challenging in the context of the pandemic are key to promoting high-quality teaching and learning practices that contribute to equitable, positive youth outcomes in STEM.