Assessing reading fluency is critical because it functions as an indicator of comprehension and overall reading achievement. Although theory and research demonstrate the importance of ORF proficiency, traditional ORF assessment practices are lacking as sensitive measures of progress for educators to make instructional decisions. The purpose of this study is to compare traditional ORF measures/administration to a computerized ORF assessment system based on speech recognition software (CORE). Using WCPM scores as the outcome, we compare: (a) traditional ORF passages to CORE passages, (b) CORE passage lengths, and (c) scoring methods. We used a general linear model with two within-subject factors, to test the mean WCPM score differences between passage length, scoring method, and their interaction. We found that CORE passages, whether short, medium, or long, tended to have higher WCPM means than the Traditional ORF passages. Real-time scoring tended to have higher WCPM means than both Audio Recording and ASR scoring types, and the ASR and Recording scores were quite similar across passage length and grade, providing preliminary evidence that the speech recognition scoring engine can score passages as well as human administrators in real settings. [Published by the Society for the Scientific Study of Reading.]