A method is described for choosing sample sizes when the domain to be sampled consists of a finite set of sentences and the purpose is to construct a test to assess the comprehension or the readability of written discourse. The testing method is that proposed in Bormuth's work on transformational analysis within a criterion-referenced measurement…

This study examined the results obtained from certification applicants and practitioners on a national certification examination for clinical laboratory scientists (medical technologists), using a modified Angoff procedure to establish the cut-off score. The major question of the investigation concerned whether the cut-off score was appropriate.…

Summarizes item review in the development of criterion-referenced tests, including logical item review, which examines the match between instructional intent and the items; empirical item review, which examines response patterns; traditional item review; and instructional sensitivity of test items. Twenty-eight references are listed. (MBR)

The author replies to Marx's critique of his essay, "Test Purpose and Item Type," which appears on pages 8-13 of this issue of "Canadian Journal of Education." For Marx's comments, see pages 14-19. (SJL)

The purpose of this study was to examine the homogeneity in difficulty of item domains and the effectiveness of Rasch pre-equating procedures for adjusting test scores for differences in the difficulty of tests constructed by sampling from item domains. The data used were taken from a field test and calibration of 810 tenth-grade items in…

As alternatives to the objectives-based approach to specifying content domains for test construction purposes, six strategies are proposed: (1) amplified objectives; (2) Instructional Objectives Exchange (IOX) test specifications; (3) item transformations; (4) item forms; (5) algorithms; and (6) mapping sentences. Their effectiveness is assessed…

Two techniques for writing achievement test items to accompany instructional materials are contrasted: writing items from statements of instructional objectives, and writing items from semi-automated rules for transforming instructional statements. Both systems resulted in about the same number of faulty items. (Author/JKS)

Item mappings are widely used in educational assessment for applications such as test administration (through test form assembly and computer assisted testing) and for criterion-referenced (CR) interpretation of test scores or scale anchoring. Item mappings are also used to construct ordered item booklets in the CTB/McGraw Hill Bookmark standard…

By noting that a Rasch or two parameter logistic (2PL) item belongs to the exponential family of random variables and that the probability density function (pdf) of the correct response (X=1) and the incorrect response (X=0) are symmetric with respect to the vertical line at the item location, it is shown that the conjugate prior for ability is…

This study addressed what standard-setting panelists think about when they make item performance estimates for a barely proficient student. This study extended previous studies by considering the factors that influenced panelists' decisions in an Angoff (1971)-based standard-setting study as a function of their item performance estimates.…

The paper presents 12 steps in developing and validating criterion referenced tests (CRTs). The author emphasizes the need to closely examine the test's stated purpose and trace the test through the 12 steps in order to find CRTs that are useful rather than useless. Examples are given for each step: preparing or selecting objectives; developing…

When determining criterion-referenced test length, problems of guessing are shown to be more serious than expected. A new method of scoring is presented that corrects for guessing without assuming that guessing is random. Empirical investigations of the procedure are examined. Test length can be substantially reduced. (Author/CM)

Three item selection methods for criterion-referenced tests are examined: the classical theory of item difficulty and item-test correlation; the latent trait theory of item characteristic curves; and a decision-theoretic approach for optimal item selection. Item contribution to the standardized expected utility of mastery testing is discussed. (CM)

The unfulfilled promise of criterion-referenced measurement is that it would permit valid inferences about what a student could and could not do. To come closest to achieving all that criterion-referenced testing originally promised, tests of higher item density, with more items per amount of domain, are required. (SLD)

Item analysis procedures appropriate for domain-referenced classroom testing are described. A conceptual framework within which item statistics can be considered and promising statistics in light of this framework are presented. The sampling fluctuations of the more promising item statistics for sample sizes comparable to the typical classroom…