The purpose of this research was to study the effect of the three different ways of increasing the number of points contributed by constructed response (CR) items on the reliability of test scores from mixed-item-format tests. The assumption of unidimensionality that underlies the accuracy of item response theory model-based standard error…

A multilevel variance component analysis from the pilot year of pretesting an instrument--the GENED--designed to present information about general education outcomes is presented, using data from about 11,000 college students. The analysis addresses the discriminant validity of the subtests and statistical issues in test construction. (SLD)

The interaction of reading and computational abilities in solving mathematics word problems was investigated with the responses of about 20,000 fourth graders and 20,000 eighth graders to items from the Iowa Tests of Basic Skills. Item responses exhibited different types of interaction, a finding with implications for item-response theory…

Concepts behind plausible values in estimating population characteristics from sparse matrix samples of item responses are discussed. The use of marginal analyses is described in the context of the National Assessment of Educational Progress, and the approach is illustrated with Scholastic Aptitude Test data for 9,075 high school seniors. (SLD)

The scales of the National Assessment of Educational Progress (NAEP), as constructed, do not yield meaningful criterion-referenced interpretations. Poorly defined NAEP goals and the present knowledge base do not allow the measurement of what examinees can and cannot do. Inappropriate interpretations of NAEP data are discussed, with specific…

A study involving 102 high school students (49 males and 53 females) from Taiwan revealed that the order of response scale labels had a primacy effect on subjects' choices of the alternatives in Likert-type attitude scales. Practical implications of the response-order effects for measurement are discussed. (SLD)

A model-testing approach for evaluating the stability of item response theory item parameter estimates (IPEs) in a pretest-posttest design is illustrated. Nineteen items from the Head Start Measures Battery were used. A moderately high degree of stability in the IPEs for 5,510 children assessed on 2 occasions was found. (TJH)

The analytically derived asymptotic standard errors (SEs) of maximum likelihood (ML) item estimates can be approximated by a mathematical function without examinees' responses to test items, and the empirically determined SEs of marginal maximum likelihood estimation (MMLE)/Bayesian item estimates can be obtained when the same set of items is…

Research on the utility of feedback to raters in the form of performance reports has produced mixed findings (Lunt, Morton, & Wigglesworth, 1994; Wigglesworth, 1993) and has thus far been trialled only in oral assessment contexts. This article reports on a study investigating raters' attitudes and responsiveness to feedback on their ratings of…

The purpose of this study was to explore how Year 8 students answered Third International Mathematics and Science Study (TIMSS) questions and whether the test questions represented the scientific understanding of these students. One hundred and seventy-seven students were tested using written test questions taken from the science test used in the…

Individual differences in measures of prereading skills and in questionnaire measures of 4-5-year-old twins' print environments in Australia, Scandinavia, and the United States were explored with a behavioral-genetic design. Modest phenotypic correlations were found between environmental measures and the twins' print knowledge, general verbal…

The assessment of children in their years before school and their first years of school has been, traditionally, informal. Further, assessment of children's mathematical skills at this level has been infrequent compared to social, emotional and physical assessments. However, there are contexts where reliable, valid, standardised data from…

This paper provides a summary of the authors' attempts to uncover links between their measures, classroom mathematics instruction, and student learning. This paper also provides evidence regarding one central critique of their measures: that multiple-choice assessments cannot validly represent the knowledge, skills, and judgment involved in actual…

This paper describes an item response theory (IRT) based method of differential item functioning (DIF) detection that involves neither separate calibration nor ability grouping. IRT is used to generate residual scores, scores free of the effects of person or group ability and item difficulty. Analysis of variance is then used to test the group…

A multidimensional item response theory (IRT) model for polytomously scored items, based on the graded response model and using the normal ogive, is developed; and an EM algorithm that may be used to estimate the parameters of the model is also discussed. The model is illustrated through a simulation study in which the polytomous item responses of…