Implementation fidelity is considered an indicator of an intervention's internal validity. In reading research, while student outcome measures are often well described, measures of implementation fidelity have received far less attention. Although researchers have become interested in the relation between implementation fidelity and student reading outcomes, measures of fidelity are seldom well described or validated prior to use. As a consequence, reading intervention researcher use a wide variety of implementation fidelity measures and a range of methods to convert observations into scores. In this study, I conducted a methodological case study to explore the effects of these different score conversion mechanisms on the rank order of participants, and multi-level model outcomes. For the analysis, I compared 10 score conversion mechanisms for dosage and 12 for adherence across three reading outcomes using data from a large state-funded initiative to implement MTSS for reading in several high-needs elementary schools. To examine possible effects, I estimated baseline models for each reading outcome and fidelity measure and compared these to models including the different score conversions mechanisms on model assumptions of linearity, homoscedasticity, and normality of distribution for the student-level residuals, model fit, LR test significance, and parameter estimates and 95% confidence interval changes. The outcomes showed scoring mechanisms for dosage did not rank participants similarly. Adherence mechanisms related to each other, but masked differences in particular scores. Using different mechanisms resulted in different model fit decisions, parameter estimates, and model assumptions. Notably, the most commonly used mechanisms did not improve models compared to the baseline. This study highlighted several issues common to the examination of implementation fidelity in reading intervention research. Researchers should give careful thought not only to the outcome measures in their studies, but also to the measurement, assumptions, and subsequent use of implementation fidelity elements. These decisions need to be part of the research design phase, since they might depend on the nature of the intervention and the intention of the use of the measure. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]