A description of the difficulties encountered in constructing a criterion-referenced test to assess end-of-year skills as part of a program evaluation is presented. The problems encountered in preparing the customized test are described in humorous detail. The first problem involved the listening test obtained from a publisher without…

Item analysis procedures appropriate for domain-referenced classroom testing are described. A conceptual framework within which item statistics can be considered and promising statistics in light of this framework are presented. The sampling fluctuations of the more promising item statistics for sample sizes comparable to the typical classroom…

Although effective and efficient assessment procedures are crucial to the success of organizational training programs, the tests used for both student and program evaluation are often developed by instructional designers (IDS) and subject matter experts (SMEs) with little formal training in test development. This paper summarizes the major…

Concern for enhancing construct validity of vocational interest measures provides a focus for scale construction quite distinct from that derived from a criterion-referenced strategy: Construct-oriented measurement implies: (1) substantive definitions of dimensions; (2) concern for internal consistency reliability, as well as generalizability; (3)…

Research on the lack of correlation between standardized tests and school district goals has suggested that district goals might be better met by supplementing standardized tests with district-made criterion-referenced testing systems. This article reports on the development of such a system, the Academic Learning Progress Assessment System…

The Maryland State Department of Education issued a request for proposals to develop a score reporting system for the Maryland Functional Testing Program. RMC Research Corporation conducted a literature review of extant literature and developed a national survey of non-norm-referenced test score reporting practices. This comprehensive analysis of…

This paper proposes that instructional validity (i.e., the extent to which teachers actually teach the skills that are tested) should be part of the test development process. The development of a district-wide ninth-grade math exam for General Mathematics I (GMI) in the Charleston County School District, South Carolina, is used to illustrate how a…

The Assessment Resource Center (ARC) at Florida State University provides computer assisted testing (CAT) for approximately 4,000 students each term. Computer capabilities permit a small proctoring staff to administer tests simultaneously to large numbers of students. Programs provide immediate feedback for students and generate a variety of…

Georgia's competency testing program has gained national recognition and has served as a model for other states planning such programs. The focus of this paper is on the Teacher Certification Test (TCT) in Administration and Supervision developed to conform to stringent legal and job-relatedness requirements. The test is required of educators…

The development of a test design methodology used to construct a criterion-referenced System Achievement Test (CR-SAT) for selected Naval enlisted classification (NEC) in the Strategic Weapon System (SWS) of the United States Navy is described. Subject matter experts, training data analysts and educational specialists developed a comprehensive…

Provides an alternative analysis of the reliability associated with the van Hiele Geometry Test based on the assumption that the norm-referenced reliability coefficients provided by the developers were inappropriate. Discusses the agreement coefficient and the kappa coefficient. (YP)

In 1984 an evaluation team examined the Maryland Functional Testing Program (MFTP), a statewide competency-based, criterion-referenced testing program that was instituted to limit awarding of high school diplomas to those students who had earned scores above cut scores on a series of achievement tests. This article examines the procedures the…

Open testing and test disclosure legislation has fostered a renewed interest in the problems and issues associated with test equating. This paper focuses on a specific application of test equating, that of professional and occupational testing for certification and licensure. Test equating is the process of making test scores from different forms…

This paper presents an evaluation of Army job training development and testing practices, with a focus on Advanced Individual Testing. Information comes from intensive interviews with school instructors and from observations in the schools. Results indicate that some aspects of the Instructional Systems Development (ISD) model have been…

This paper describes the development of standardized evaluation procedures for Pasco County's inservice teacher education. Since 1983, legislation requires teacher education centers in Florida to report attainment data based on measurable objectives. Using a classification system adapted from the "Rand Change Agent Study," directors and…