Examines four factors involved in the determination of how many test items should be constructed or sampled for a set of objectives: (1) the type of decision to be made with results, (2) importance of objectives, (3) number of objectives, and (4) practical constraints. Specific guidelines that teachers and evaluators can use and an illustrative…

Seventeen statistics for measuring the reliability of criterion-referenced tests were critically reviewed. The review was organized into two sections: (1) a discussion of preliminary considerations to provide a foundation for choosing the appropriate category of "reliability" (threshold loss function, squared-error loss-function, or…

Two approaches to criterion-referenced measurement are described and contrasted--domain-referenced testing and mastery testing. This paper is organized according to ten issues or stages in test construction: (1) content domain specification; (2) item construction; (3) item domain specification; (4) item analysis; (5) item selection; (6) parallel…

A method is described for choosing sample sizes when the domain to be sampled consists of a finite set of sentences and the purpose is to construct a test to assess the comprehension or the readability of written discourse. The testing method is that proposed in Bormuth's work on transformational analysis within a criterion-referenced measurement…

As alternatives to the objectives-based approach to specifying content domains for test construction purposes, six strategies are proposed: (1) amplified objectives; (2) Instructional Objectives Exchange (IOX) test specifications; (3) item transformations; (4) item forms; (5) algorithms; and (6) mapping sentences. Their effectiveness is assessed…

A dozen different approaches that yield 13 reliability indices for criterion-referenced tests were identified and grouped into three categories: threshold loss function, squared-error loss function, and domain score estimation. Indices were evaluated within each category. (Author/RL)

Attempts to select empirically the optimal cutting score or criterion level for a test based on response data from validation samples of instructed and uninstructed students. This score maximizes the probability of correct mastery-nonmastery decisions (or minimizes the probability of incorrect decisions). (Author/RK)

Sixteen item statistics recommended for use in the development of criterion-referenced tests were evaluated. There were two major criteria: (1) practicability in terms of ease of computation and interpretation and (2) meaningfulness in the context of the development process. Most of the statistics were based on a comparison of performance changes…

Thirty-eight methods are presented for either setting standards or adjusting them based on an analysis of classification error rates. A trilevel classification scheme is used to categorize the methods, and 10 criteria of technical adequacy and practicability are proposed to evaluate them. (Author/LMO)

Four factors essential to determining how many items should be constructed or sampled for a set of objectives are examined: (1) importance and type of decisions to be made with the results; (2) importance and emphases assigned to the instructional and behavioral objectives; (3) number of objectives; (4) practical constraints, such as item writing…