This paper presents an evaluation of Army job training development and testing practices, with a focus on Advanced Individual Testing. Information comes from intensive interviews with school instructors and from observations in the schools. Results indicate that some aspects of the Instructional Systems Development (ISD) model have been…

This issue of Measurement in Education is presented in the form of a dialogue between Dr. Robert L. Ebel, Distinguished Professor of Educational Measurement at Michigan State University, and Dr. Samual A. Livingston, Program Research Scientist at the Educational Testing Service. Alternative views on some aspects of the use of tests in assessing…

In light of the widespread use of competency testing, the authors consider that it is important to determine ways of developing and using competency testing to insure that it achieves its full potential. The paper, in three parts, introduces a model for the development and validation of competency tests, reviews several methods for setting…

A large number of seemingly diverse coefficients have been proposed as indices of dependability, or reliability, for domain-referenced and/or mastery tests. In this paper, it is shown that most of these indices are special cases of two generalized indices of agreement: one that is corrected for chance, and one that is not. The special cases of…

From 1974 through 1978, three methods of assessing Scottish high school students' (O level) English achievement were studied: (1) Ordinary (O level) examinations; (2) assessment of writing skills (folio assessment); and (3) criterion referenced tests developed by the Scottish Council for Research in Education (SCRE) to measure the objectives of…

Technical considerations associated with item selection and reliability assessment are considered in relation to criterion-referenced tests constructed to provide group information. The purpose is to emphasize test building and the evaluation of test scores in program evaluation studies. It is stressed that an evaluator employ a performance or…

Cross-referenced to instructional objectives defined in a previous publication, (Guidelines to Mathematics, K-6), the Wisconsin Mathematics Test can be used to measure achievement gains over a given year, or to detect student strengths and weaknesses, by correlating missed items with the objectives. It is also possible to compare results on a…

The feasibility of generating multiple-choice test questions by transforming sentences from prose instructional materials was examined. A computer-based algorithm was used to analyze prose subject matter and to identify high-information words. Sentences containing selected words were then transformed into multiple-choice items by four writers who…

The feasibility of using criterion referenced tests (CRTs) in a large-scale evaluation conducted in an effectiveness evaluation context was investigated. The study began by examining the theory that structures the development and validation of CRTs to discover whether, on theoretical grounds alone, CRTs are suitable or not suitable for large-scale…

Part I of this report attempts to describe the system that was developed for local educational agencies by the Delaware State Department of Public Instruction to support classroom and curricular improvement in mathematics through the administration of an objective-referenced test in mathematics to grade four students. This system includes the…

Examines the Decoding Skills Test (DST), a criterion referenced, diagnostic instrument designed to identify deficiencies in word decoding. (JC)

Based on the standard setting procedures or the American Board of Preventive Medicine for their Core Test, this article describes how Rasch measurement can facilitate using test content judgments in setting a standard. Rasch measurement can then be used to evaluate and improve the precision of the standard and to hold it constant across time.…

The Early Learning Accomplishment Profile (Early LAP) provides a systematic method for observing children's functioning in the birth to 36-month age range in order to assist teachers, clinicians, and parents in assessing individual skills development in six developmental domains: gross motor, fine motor, cognition, language, self-help, and social…

A latent trait method is presented to investigate the possibility that Angoff or Nedelsky judges specify inconsistent probabilities in standard setting techniques for objectives-based instructional programs. It is suggested that judges frequently specify a low probability of success for an easy item but a large probability for a hard item. The…

The purpose of the investigation was to obtain some relationships among (1) test lengths, (2) shape of domain-score distributions, (3) advancement scores, and (4) several criterion-referenced test score reliability and validity indices. The study was conducted using computer simulation methods. The values of variables under study were set to be…