This study was patterned after a previous study by Skaggs and Lissitz (1992) in which inconsistency of differential item functioning (DIF) was reported across test administrations. They suggested multidimensionality of test data as one possible reason for inconsistency. Therefore, in this study, DIF indices which were developed recently with a…

This paper discusses the use of two theories of item analysis and test construction, their strengths and weaknesses, and applications to the design of the Hawaii State Test of Essential Competencies (HSTEC). Traditional analyses of the data collected from the HSTEC field test were viewed from the perspectives of item difficulty levels and item…

This study examines the accuracy of the self-scoring efforts of 306 eighth-graders on the Kuder General Interest Survey (GIS), and suggests possible methods to improve self-scoring accuracy. The GIS is widely used to assist junior high school students with their educational and vocational planning. After the administration of the test by English…

The effects of students taking different levels of the same standardized achievement test were assessed by administering two levels of the same test to each student. The functional level of the test was taken by all students. The second level of testing was randomly assigned at the adjacent higher or lower level of the test. Functional level…

An example application of portfolio assessment was developed and the model and estimates of reliability derived from the literature were then used to estimate the characteristics of an operational large-scale portfolio assessment program. Costs were estimated to put results in a realistic context. (SLD)

In England and Wales, a National Curriculum initiated in 1988 was designed to ensure that all schools provided a curriculum which represented different areas of knowledge. The past 20 years has increasingly seen more emphasis on the link between the financial amounts spent on education and subsequent return on this money. The impact of the…

Content integration in performance assessment involves mixing different areas of knowledge in one assessment. In this type of testing situation, assessment tasks are designed to measure the ability of students to solve problems by applying their knowledge and skills in multiple content areas. This study examined the effect of integrated science…

Political and legislative pressures have posed a number of measurement issues and challenges to the development of sound, valid voluntary national tests (VNTs). This paper focuses on what appear to be the most difficult technical issues related to the VNT proposed by President Clinton in 1997. Technical issues refer to psychometric issues, as…

Every year approximately 1,300 first-year students at the Rochester Institute of Technology complete a 50-minute placement essay during summer and fall orientations. The essays are scored holistically, and the students are placed into one of three levels of an English composition course. At the end of the 10-week quarter of instruction, students…

The consequences of large state testing are often uniformity of expectations for achievement. The largest impact of higher standards, then, are realized by traditionally disenfranchised student populations, particularly the least affluent who are most likely to bear the yoke of low expectation. This paper advances S. Messick's (1981) fundamental…

This paper presents an evaluation of Army job training development and testing practices, with a focus on Advanced Individual Testing. Information comes from intensive interviews with school instructors and from observations in the schools. Results indicate that some aspects of the Instructional Systems Development (ISD) model have been…

This paper deals with the problems that arise in performance assessment from the granularity that results from having a small number of tasks or prompts and raters of responses to these tasks or prompts. Two problems are discussed in detail: (1) achieving a satisfactory degree of reliability; and (2) equating or adjusting for differences of…

The statewide teaching performance assessment instruments being used in Georgia, North Carolina, and Florida were examined. Forty-one reliability and validity studies regarding the instruments in use in each state were collected from state departments and universities. Georgia uses the Georgia Teacher Performance Assessment Instrument. North…

The necessity of controlling the order in which trained essay raters for a statewide writing assessment program receive student essays was studied. The underlying theoretical question concerns possible rater bias caused by raters reading long strings of essays of homogeneous quality; this problem is usually referred to as context effect or…

Language testing historians have tended to ignore a significant period in the evolution of language tests, the years 1883-1929. In the earliest years, testing focused on knowledge about, not of, the language and reflected the teaching of Latin and Greek more than that of living languages. Grammatical formalism and translation were emphasized, and…