Previous investigations of the ability of content experts and test developers to estimate item difficulty have, for themost part, produced disappointing results. These investigations were based on a noncomparative method of independently rating the difficulty of items. In this article, we argue that, by eliciting comparative judgments of…

It is so easy to criticize the SAT that most observers overlook the weaknesses of its architect, the College Board. This author contents that, until the latter is replaced, however, the former will never be fixed. The College Board has every incentive to create a complex, stressful, expensive college admissions system. Because it is accountable to…

Views on testing--its purpose and uses and how its data are analyzed--are related to one's perspective on test takers. Test takers can be viewed as learners, examinees, or contestants. I briefly discuss the perspective of test takers as learners. I maintain that much of psychometrics views test takers as examinees. I discuss test takers as a…

The current study examines how using a multidimensional battery of predictors (high-school grade point average (GPA), SAT/ACT, and biodata), and weighting the predictors based on the different values institutions place on various student performance dimensions (college GPA, organizational citizenship behaviors (OCBs), and behaviorally anchored…

Tests of college learning are often administered to obtain value-added scores indicating whether score gains are below, near, or above typical performance for students of given entering academic ability. This study compares the qualities of value-added scores generated by the original Collegiate Learning Assessment value-added approach and a new…

Case-specificity, i.e., variability of a subject's performance across cases, has been a consistent finding in medical education. It has important implications for assessment validity and reliability. Its root causes remain a matter of discussion. One hypothesis, content-specificity, links variability of performance to variable levels of relevant…

Focusing on writing placement at a particular university, the authors analyze the limits of SAT tests as a tool in this process. They then describe the writing program's adoption of a supplementary measure: a faculty committee's review of essays by students who may need to be reassigned to a different writing course. They describe how and why a…

The attribute hierarchy method (AHM) is a psychometric procedure for classifying examinees' test item responses into a set of structured attribute patterns associated with different components from a cognitive model of task performance. Results from an AHM analysis yield information on examinees' cognitive strengths and weaknesses. Hence, the AHM…

This article examines the influence of keyboarding versus handwriting in a high-stakes writing assessment. Conclusions are based on data collected from a pilot project to move Old Dominion University's Exit Exam of Writing Proficiency from a handwritten format into a dual-option format (i.e., the students may choose to handwrite or keyboard the…

How do scores from different tests relate to each other? Three types of score linkage are discussed: equating, concordance, and prediction of expected scores. Statistical indices, in conjunction with rational considerations, are needed to determine whether the highest level of linkage attainable between scores from two "tests" is the…

This study examined to what extent, if any, estimates of reliability for a multiple choice test are affected by the presence of large item sets where each set shares common reading material. The purpose of this research was to assess the effect of local item dependence on estimates of reliability for verbal portions of seven forms of the old and…

A study of Scholastic Aptitude Test scores for nine groups of students with disabilities taking special test administrations found differences in score levels among disability groups but no significant differences of measurement precision and no evidence of disadvantage for disabled students. (Author/MSE)

It is sometimes sensible to think of the fundamental unit of test construction as being larger than an individual item. This unit, dubbed the testlet, must pass muster in the same way that items do. One criterion of a good item is the absence of differential item functioning (DIF). The item must function in the same way as all important…

The contribution of the Scholastic Aptitude Test (SAT) to education is described, and the most frequently asked questions about its use are addressed. The SAT became a prominent tool in college admissions when the members of the College Board recognized in the 1930's that the Board's program of very specific subject-matter tests was no longer…

Previous research on differences in the reliability, validity, and difficulty of essay tests given under different timing conditions has indicated that giving examinees more time to complete an essay may raise their scores to a certain extent, but does not change the meaning of those scores, or the rank ordering of students. There is no evidence…