Multiple-choice questions (MCQs) are widely used in digital learning systems, as they allow for automating the assessment process. However, owing to the increased digital literacy of students and the advent of social media platforms, MCQ tests are widely shared online, and teachers are continuously challenged to create new questions, which is an…

It is imperative to collect validity evidence prior to interpreting and using test scores. During the process of collecting validity evidence, test developers should consider whether test scores are contaminated by sources of extraneous information. This is referred to as construct irrelevant variance, or the "degree to which test scores are…

A conceptualization of multiple-choice exams in terms of signal detection theory (SDT) leads to simple measures of item difficulty and item discrimination that are closely related to, but also distinct from, those used in classical item analysis (CIA). The theory defines a "true split," depending on whether or not examinees know an item,…

Chemistry faculty members are highly skilled in obtaining, analyzing, and interpreting physical measurements, but often they are less skilled in measuring student learning. This work provides guidance for chemistry faculty from the research literature on multiple-choice item development in chemistry. Areas covered include content, stem, and…

The general purpose of this study was to determine the effects of similarities between stems and keyed choices on test difficulty. Unlike previous investigations of this undesirable characteristic of some multiple-choice items, the present study employed items that were unintentionally faulty and samples of examinees who were highly experienced…

Two studies examined the effect of making the correct answer of a multiple choice test item grammatically consistent with the item. American College Testing Assessment experimental items were constructed to investigate grammatical compliance to investigate grammatical compliance for plural-singular and vowel-consonant agreement. Results suggest…

The effects of violating four item construction principles were examined to assess the validity of the principles and the importance of students' test wiseness. While flawed items were significantly less difficult than sound items, differences in item discrimination, test reliability, and concurrent validity were not observed. (Author/BW)

Nineteen multiple-choice nursing tests containing 673 items were analyzed for test wiseness, irrelevant difficulty in stem or option, linguistic/structural bias, or cultural bias. Twenty-eight types of flaws occurred at least 10 times each. (Contains 28 references.) (SK)

The quality of mathematics multiple choice items and their susceptibility to test wiseness were examined. Test wiseness was defined as "a subject's capacity to utilize the characteristics and formats of the test and/or test taking situation to receive a high score." The study used results of the Graduate Record Examinations Aptitude Test (GRE) and…

Data available from the Fall and Spring administration of the Stanford Achievement Test: Intermediate I: Form X and the Otis-Lennon Mental Ability Test for all Title I pupils in the State of New Hampshire plus a random sample for the entire state, made possible the item-by-item comparison of Fall and Spring performance on the same groups of…

This paper describes the development and investigation of a new approach to determining the existence of bias in multiple-choice test scores. Previous work in this area has concentrated almost exclusively on bias attributable to specific test items or to differences in test score distributions across racial or ethnic groups. In contrast, the…

Gives examples of major sources of unreliability in multiple choice items in health professions classroom achievement tests (clues to response combinations, mutually exclusive alternatives, implausible distractors) and offers some suggestions for eliminating them when writing such tests. (MF)