Despite prevalent use and practical application, the current and standard assessment of oral reading fluency (ORF) presents considerable limitations which reduces its validity in estimating growth and monitoring student progress, including: (a) high cost of implementation; (b) tenuous passage equivalence; and (c) bias, large standard error, and tenuous reliability. To address these limitations, the Computerized Oral Reading Evaluation (CORE) system contains an automated scoring algorithm based on a speech recognition engine and a novel latent variable psychometric model. The purpose of this study is to investigate potential student and passage predictors of low agreement between an automated speech recognition (ASR) engine and human scores of words read correctly in student oral reading fluency passages. We fit a cross-classified, variable exposure Poisson model to estimate agreement and found that the majority of variance was found at the student and recording levels, and that student demographic variables explained only a small amount (13%) of the student-level variance. [Poster presented at the National Council on Measurement in Education Annual Meeting (Apr 2017).]