Scores from computer-based tests are increasingly used to inform a variety of school- and student-level decisions. An underlying assumption is that the associated scores represent effortful responding by each student with respect to the tasks presented. An innovative method for examining evidence for this assumption involves an examination of item…

The New York State Education Department (NYSED) has a partnership with NWEA for the development of the 2024 Grades 3-8 English Language Arts Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability of the New York State Testing Program (NYSTP). The 2024 Grades 6 and 7 English Language…

In response to a request for additional analyses, in particular reporting confidence intervals around the results, we re-analyzed the data from prior studies. This supplementary report presents the results of the additional analyses addressing classification accuracy, reliability, and criterion-related validity evidence. For ease of reference, we…

In response to a request for additional analyses, in particular reporting confidence intervals around the results, we re-analyzed the data from prior studies. This supplementary report presents the results of the additional analyses addressing classification accuracy, reliability, and criterion-related validity evidence. For ease of reference, we…

This in-brief technical report documents the results from two different analytic approaches for examining the reliability of the slope for easyCBM® reading measures in Grades K-8. Results varied by grade, assessment measure, and the analytic approach. Results patterns are discussed.

We evaluated the reliability and validity of two oral reading fluency scores for 1-minute equated passages: median score and mean score. These scores were calculated from measures of reading fluency administered up to five times over the school year to students in grades six to eight (n = 1,317). Both scores were highly reliable with strong…

The present paper addresses the continuous need for methodological reflection on how to validate inferences made on the basis of test scores. Validation is a process that requires many lines of evidence. In this article we discuss the potential of eye tracking methodology in process-oriented reading test validation. Methodological considerations…

Maze tasks have appealing properties as progress-monitoring tools, but there is a need for a thorough examination of the psychometric properties of Maze tasks among middle school students. We evaluated form effects, reliability, validity, and practice effects of Maze among students in Grades 6 through 8. We administered the same (familiar) and…

Development of the Common Core State Standards (CCSS), and the creation of the Smarter Balanced Assessment Consortium (Smarter Balanced) and the Partnership for Assessment of Readiness for College and Careers (PARCC), changes the pattern of accountability testing. These changes raise the question: "How should NAEP's validity and utility be…

This article reviews the "Developmental Reading Assessment--Second Edition" (DRA2), a teacher-administered assessment that identifies students' instructional level, along with their strengths and weaknesses in reading. Once teachers calculate and interpret scores, the data can purportedly be used to inform, and possibly individualize,…

This study was designed to investigate the validity of four different aloud reading comprehension assessment measures: Maze, comprehension questions, Maze accurate response rate (MARR), and reading comprehension rate (RCR). The criterion measures used in this study were the Woodcock-Johnson III Tests of Achievement (WJ-III ACH) Broad Reading…

Item response theory (IRT) methods are generally used to create score scales for large-scale tests. Research has shown that IRT scales are stable across groups and over time. Most studies have focused on items that are dichotomously scored. Now Rasch and other IRT models are used to create scales for tests that include polytomously scored items.…

This technical report presents results from a cross-validation study designed to identify optimal cut scores when using easyCBM[R] reading tests in Oregon. The cross-validation study analyzes data from the 2009-2010 academic year for easyCBM[R] reading measures. A sample of approximately 2,000 students per grade, randomly split into two groups of…

There has been a recent emphasis on improving the academic performance of students with emotional and behavioral disorders (EBD). Improving the academic performance of students with EBD is especially important in the current accountability era in which there is much emphasis placed on performance of standardized tests. The purpose of this study…

The purpose of this study was to develop and gather validity evidence for silent reading fluency passages. A number of passages were written following a traditional story grammar structure (character, setting, events) and placed on a computer for students to read silently. We describe in detail, the manner in which content-related evidence was…