Livingston's reliability coefficients and Harris' indices of efficiency were computed along with the classical internal consistency coefficients, KR-20's (Kuder-Richardson internal consistency coefficient), for 678 criterion-referenced tests in the A through E levels of an individualized mathematics program. The coefficients were carefully studied…

Data required for evaluating a Criterion Referenced Measurement (CRM) is described with a matrix. The information within the matrix consists of the "pass-fail" decisions of two CRMs. By differentially defining these two CRMs, different concepts of reliability and validity can be examined. Indices suggested for analyzing the matrix are listed with…

Using the assumption of randomly parallel tests and concepts from generalizability theory, three signal/noise ratios for domain-referenced tests are developed, discussed, and compared. The three ratios have the same noise but different signals depending upon the kind of decision to be made as a result of measurement. (Author/JKS)

Spanish speaking first graders were administered the Artes de Lenguage (ADL)--a Spanish, criterion-referenced, language arts test. Reliability analyses indicated the adequacy of three of the four subscales (Phonetic Analysis, Vocabulary Development, Comprehension Skills, and General Skills). A principal factors analysis of the intercorrelation…

The presently viewed role and utility of measures of instructional sensitivity are summarized. A case is made that the rationale for the assessment of instructional sensitivity can be applied to all achievement tests and should not be restricted to criterion-referenced mastery tests. (Author/BW)

In this comment a recent attempt by Samuel A. Livingston to develop a theory of reliability for criterion-referenced measures is critiqued. For Livingston's rejoinder see TM 500 560. (Authors/MB)

A new method was developed to assist in the selection of a test length by utilizing computer simulation procedures and item response theory. A demonstration of the method presents results which address the influences of item pool heterogeneity matched to the objectives of interest and the method of item selection. (Author/PN)

Addressing Glass' argument (EJ 198 842) that a lack of interrelater reliability is an inherent deficiency in the Nedelsky technique, poor rater training and the need for a group decision procedure are presented as standard setting problems. (CM)

This study compared the reliability and validity indexes of randomly parallel tests administered under inclusion, exclusion, and correction for guessing directions, using 54 graduate students. It also compared the criterion-referenced grading decisions based on the different scoring methods. (TJH)

Concern for enhancing construct validity of vocational interest measures provides a focus for scale construction quite distinct from that derived from a criterion-referenced strategy: Construct-oriented measurement implies: (1) substantive definitions of dimensions; (2) concern for internal consistency reliability, as well as generalizability; (3)…

An overview of the articles and definition of terms for the journal topic of domain reference testing. (HB)

A behavioral domain is well-defined when it is clear to both test developers and test users which categories of performance should or should not be considered for potential test items. Only those tests that are keyed to well-defined domains meet the definition of criterion-referenced tests. The greatest proliferation of criterion-referenced tests…

A series of studies on conceptual and design problems in competency-based measurements are explained. The concept of validity within the context of criterion-referenced measurement is reviewed. The authors believe validation should be viewed as a process rather than an end product. It is the process of marshalling evidence to support…

Subkoviak suggested a technique for estimating both group reliability and the reliability associated with assigning a given individual to a mastery or non-mastery category based on a single test administration. Two assumptions underlie this model. First, it is assumed that had successive test administrations occurred, scores for each individual…

Present definitions of the criterion-referenced test are discussed, insufficiences noted, and a new definition proposed. Some examples of criterion-referenced tests are examined and used to educe some general principles for the development of such tests. The utility of item form processes is assessed. It is suggested that the difficulty of…