The current study examined the reliability of The Reading League Curriculum-Evaluation Guidelines (CEGs), which were developed to help school-based teams rate the presence of red flags when considering adopting specific literacy curricula. Coders (n = 30) independently used the CEGs to evaluate a free online English language arts curriculum. The results indicated strong internal consistency ([alpha] = 0.96) and high interrater reliability (H[subscript M] = 0.91, 95% CI = 0.89 to 0.93, p < 0.01). Overall, the CEGs hold the potential as a psychometrically sound tool for evaluating reading curricula. Limitations and implications for practice and research are discussed.