Background: Implementation quality is a significant factor in determining whether a program has its intended impact (Durlak and DuPre, 2008; List et al., 2021). Therefore, when studying a program's efficacy, researchers must ensure that the program is implemented as intended to estimate its impact. To evaluate whether a program is being implemented properly in the classroom during a study, researchers generally conduct classroom observations or rely on teacher self-reports (Dart et al., 2012; Schoenwald et. al., 2011). However, live observations are time-consuming and costly because they require multiple observations for each teacher to reliably estimate fidelity metrics (Cross et al., 2015; Gresham, et. al, 2017). Although self-reposts require fewer resources, they are less reliable (Gresham et al., 2017; Lee et al., 2008). In this study, we explore how data automatically collected by online learning platforms can be leveraged to evaluate implementations and how these data can help us understand who has access to high-fidelity environments. Purpose/Objective/Research Question: (1) Which variables should be used as metrics of students' dosage and engagement across conditions? (2) How much of the variance in student-level dosage and engagement metrics are associated with the students' implementation contexts? (3) Can we classify high and low-fidelity implementation contexts using the dosage and engagement metrics? (4) Do students of different demographics receive different levels of fidelity? Setting: The analyses for the current study used data from a randomized control trial described in Section 3. This study employed student-level randomization into the following programs: two gamified conditions (OLP1 and OLP2), OLP3 of traditional problems with immediate feedback, and OLP3 without immediate feedback (active control). Population: Initially, 52 seventh-grade mathematics teachers from 11 schools were recruited to participate in the study from a large, suburban district in the Southeastern United States during the summer of 2020. The district provided demographic data for the 3,434 participating students. Of the analytic sample, 48.88% of the students were identified as White, 27.14% as Asian, 14.04% as Hispanic, 4.86% as Black, 0.69% as Native American, 0.04% as Pacific Islander, and 2.98% as two or more races. The district did not provide data for 1.37% of the students. About half the students were identified by the district as female (48.14%) and male (51.81%), with less than 1% of the population not reporting gender. The district did not provide any data on non-binary students. About one-tenth (9.10%) of students had an Individualized Education Plan. Programs: During the school year, teachers were expected to provide students with 13 half-hour minute sessions to complete 30 assignments within their respective OLP. Each OLP used a different method of teaching algebraic knowledge. The first, sixth, twelfth, and thirteenth sessions were 40-45 minute assessments of their algebra knowledge and attitudes toward math. Sessions 2-5 and 7-11 were activities in their assigned technology. In both the assessments and the activity sessions, students' usage and progress for each program were captured by their respective technologies. As students solved problems in the systems, the platforms also logged information about students' actions and performance. Design: To create metrics of student dosage and engagement (RQ1), we first calculated variables from the log-file data of each program, conducted exploratory data analysis, and standardized fidelity metrics within programs. Variables identified as suitable measures of dosage and engagement across programs in RQ1 are used to produce fidelity Setting: The analyses for the current study used data from a randomized control trial described in Section 3. This study employed student-level randomization into the following programs: two gamified conditions (OLP1 and OLP2), OLP3 of traditional problems with immediate feedback, and OLP3 without immediate feedback (active control). Population: Initially, 52 seventh-grade mathematics teachers from 11 schools were recruited to participate in the study from a large, suburban district in the Southeastern United States during the summer of 2020. The district provided demographic data for the 3,434 participating students. Of the analytic sample, 48.88% of the students were identified as White, 27.14% as Asian, 14.04% as Hispanic, 4.86% as Black, 0.69% as Native American, 0.04% as Pacific Islander, and 2.98% as two or more races. The district did not provide data for 1.37% of the students. About half the students were identified by the district as female (48.14%) and male (51.81%), with less than 1% of the population not reporting gender. The district did not provide any data on non-binary students. About one-tenth (9.10%) of students had an Individualized Education Plan. Programs: During the school year, teachers were expected to provide students with 13 half-hour minute sessions to complete 30 assignments within their respective OLP. Each OLP used a different method of teaching algebraic knowledge. The first, sixth, twelfth, and thirteenth sessions were 40-45 minute assessments of their algebra knowledge and attitudes toward math. Sessions 2-5 and 7-11 were activities in their assigned technology. In both the assessments and the activity sessions, students' usage and progress for each program were captured by their respective technologies. As students solved problems in the systems, the platforms also logged information about students' actions and performance. Design: To create metrics of student dosage and engagement (RQ1), we first calculated variables from the log-file data of each program, conducted exploratory data analysis, and standardized fidelity metrics within programs. Variables identified as suitable measures of dosage and engagement across programs in RQ1 are used to produce fidelity metrics which are then used in subsequent analysis to address RQ2 and RQ3. Next, to evaluate how much of the dosage and engagement metrics variance is associated with components of the implementation context (RQ2), we estimated a series of multilevel models. Each model included random intercepts for school, teacher, and class regressed on the fidelity metrics identified and standardized for RQ1, and computed ICCs to evaluate the importance of the students' contexts in predicting their dosage and engagement. Next, we conducted a latent profile analysis to identify different fidelity profiles. We aggregated the fidelity metrics identified in RQ1 at the context level (school, teacher, or class) with the highest ICCs from the analysis for RQ2. Finally, to understand whether their students had different opportunities to be in high-fidelity contexts, we estimated logistic regression models regressing whether or not the students were in a high-fidelity context on their demographic data. Data Collection: Students' log-file data were collected by their respective programs. The researchers did not have access to the OLP2 database; however, before and after students solved problems in OLP2, they logged into OLP3 and answered a series of self-reported questions about their starting and ending progress in the program. All data from this study are available through the Open Science Foundation (https://osf.io/r3nf2/). Variables included in the analyses of fidelity are presented in Table 1. Results: Three metrics emerged as best suited to evaluate the students' dosage and engagement in their respective programs: the percentage of assignments students started, the total number of problems they attempted during the study, and their average problems completed per assignment (Table 2). Teachers had the highest intraclass correlations on each fidelity metric, suggesting that in this study, the teachers were the most important contextual factor in determining implementation fidelity (Table 3). Next, we identified two latent profiles of teachers: high and low fidelity (Figure 1). Finally, we found that students who were not White or Asian and those with Individualized Education Plans (IEPs) were significantly less likely to be taught by high-fidelity teachers compared with their peers (Table 4), yet these effects went away after accounting for students pretest algebraic knowledge scores. These findings reveal bias in access to high-fidelity implementations, where lower-performing students are less likely to have access, and these students are more likely to be students of color. These findings highlight the importance of implementation fidelity monitoring, and future work should how to ensure equitable access to high-quality programs in high-fidelity environments.