As the use of process data in large-scale educational assessments is becoming more common, it is clear that data on examinees' test-taking behaviors can illuminate their performance, and can have crucial ramifications concerning assessments' validity. A thorough review of the literature in the field may inform researchers and practitioners of…

The central purpose of this paper is to examine validity problems arising from the multiple-choice items and technical passages in the Test of English as a Foreign Language Internet-based Test (TOEFL iBT) reading section, primarily concentrating on construct-irrelevant variance (Messick, 1989). My personal TOEFL iBT experience, along with my…

For test scores that rely on the accurate estimation of ability via an IRT model, their use and interpretation is dependent upon the assumption that the IRT model fits the data. Examinees who do not put forth full effort in answering test questions, have prior knowledge of test content, or do not approach a test with the intent of answering…

In most large-scale programs of tests that aid in making high-stakes decisions, such as the "TOEIC"® family of products and service, it is not unusual for a significant portion of test takers to retake the test at multiple times.The study reported here used multilevel growth modeling to explore the score change patterns of nearly 20,000…

In the arena of educational testing, accessibility refers to the degree to which students are given the opportunity to participate in and engage a test. Accessibility theory is a model for examining the interactions between the test-taker and the test itself and defining how they may decrease some students' access to the test event, ultimately…

Reports on introspective and retrospective accounts from test takers, in an attempt to examine the validity of reading and test-taking "skills." It is noted that there are clearly certain skills or processes involved in answering test items in a particular format that are not necessarily specified by the test writer. (15 references) (GLR)

Two studies examined the effect of making the correct answer of a multiple choice test item grammatically consistent with the item. American College Testing Assessment experimental items were constructed to investigate grammatical compliance to investigate grammatical compliance for plural-singular and vowel-consonant agreement. Results suggest…

A critique of a variety of formats used in combined-response test items (those in which the respondent must choose the correct combination of options: a and b, all of the above, etc.) illustrates why this kind of testing is inherently flawed and should not be used in optometry examinations. (MSE)

The effects of violating four item construction principles were examined to assess the validity of the principles and the importance of students' test wiseness. While flawed items were significantly less difficult than sound items, differences in item discrimination, test reliability, and concurrent validity were not observed. (Author/BW)

A study investigated possible reasons for the low performance in 2000 on the writing portion of the Delaware Student Testing Program (DSTP) by students, especially in grades 3 and 5. The study also investigated ways to improve classroom instruction in writing. A panel of teachers reviewed the anchor papers and the process of testing. Panel members…

Two studies examined the extent to which test takers use plausibility as a method for locating correct responses when guessing and the extent to which scores can be improved by teaching test takers this approach. Results confirm that this aspect of multiple choice items merits further consideration by test constructors. (Author/BW)

The objectives for this study were to: (1) develop a valid, reliable measure of test-wiseness with equivalent forms for use with students in the health sciences; and (2) determine the level of test-wiseness of entering medical students. The test-wiseness areas included in this study were: similar options, umbrella term, item give-away, convergence…

This article discusses the validity issue in teacher-made tests. Seventh-grade students' comments about their responses to a test designed to illustrate faulty items suggests students are quite proficient in using secondary clues to figure out correct answers. Teacher comments suggest teachers are unaware they provide such clues. (Author/JAZ)

This experimental study was conducted to: (1) provide further information on the susceptibility to special preparation of three Graduate Record Examination analytical item types; (2) determine the efficacy of self-study test familiarization materials for these types; and (3) ascertain the effects of several different components of special…

The quality of mathematics multiple choice items and their susceptibility to test wiseness were examined. Test wiseness was defined as "a subject's capacity to utilize the characteristics and formats of the test and/or test taking situation to receive a high score." The study used results of the Graduate Record Examinations Aptitude Test (GRE) and…