Social scientists have long agreed that satisficing behavior increases error and reduces the validity of survey data. There have been numerous reviews on detecting satisficing behavior, but preventing this behavior has received less attention. The current narrative review provides empirically supported guidance on preventing satisficing by…

Questionnaires in educational research assessing students' attitudes and beliefs are low-stakes for the students. As a consequence, students might not always consistently respond to a questionnaire scale but instead provide more random response patterns with no clear link to items' contents. We study inter-individual differences in students'…

Retrospective pretests ask respondents to report after an intervention on their aptitudes, knowledge, or beliefs before the intervention. A primary reason to administer a retrospective pretest is that in some situations, program participants may over the course of an intervention revise or recalibrate their prior understanding of program content,…

Formative assessments are an important component of massive open online courses (MOOCs), online courses with open access and unlimited student participation. Accurate conclusions on students' proficiency via formative, however, face several challenges: (a) students are typically allowed to make several attempts; and (b) student performance might…

In undergraduate research settings, students are likely to encounter anomalous data, that is, data that do not meet their expectations. Most of the research that directly or indirectly captures the role of anomalous data in research settings uses post-hoc reflective interviews or surveys. These data collection approaches focus on recall of past…

Inattentiveness and social desirability might be particularly problematic for self-report scales in creativity and arts research. Respondents who are inattentive or who present themselves favorably will score highly on scales that yield positively skewed distributions and that assess socially valued constructs, such as scales measuring creativity…

When dealing with missing responses, two types of omissions can be discerned: items can be skipped or not reached by the test taker. When the occurrence of these omissions is related to the proficiency process the missingness is nonignorable. The purpose of this article is to present a tree-based IRT framework for modeling responses and omissions…

The overall purpose of the research studies described in this report was to investigate the quality of the administration of the Partnership for Assessment of Readiness for College and Careers (PARCC) assessment during its first operational year (2014-2015). Findings from these studies can be used to identify threats to the validity of PARCC test…

Proper survey design is essential to obtain reliable, replicable data from research subjects. One threat to inferences drawn from surveys is anchoring-and-adjusting. Tversky and Kahnemann (1974) observed that participants' responses to questions depended systematically on irrelevant information they received prior to answering. It is important for…

The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method against the oral examination (OE) method. MCQs are widely used and their importance seems likely to grow, due to their inherent suitability for electronic assessment. However, MCQs are influenced by the tendency of examinees to guess…

Reviews randomized response technique (RRT), survey method developed to reduce or eliminate underreporting of sensitive behaviors. Describes RRT models successfully used in application studies, considers issues important in using various RRT models, and illustrates their use with some realistic data examples. Discusses techniques for reducing…

Contends that the Speech Sounds Perception Test form (Adult and Midrange versions) is structured such that correct responses can be determined rationally. If a patient identifies and responds according to that structure, the validity of the test is compromised. Posttest interview is suggested as a simple solution. (Author/JAC)

Categories of questionnaire response sets and the ability to interpret response differences as related to soliciting student attitudes. (DAG)

Indices reflecting the probability that the observed similarity between the multiple-choice test responses of two examinees was due to chance are presented. Applications are presented for apprehending cheaters and for evaluating the prevalence of this form of cheating on an examination. (Author/JKS)

The influence of scale format on results was examined. Two Likert type formats, one with all choice points defined and one with only end-points defined, were administered. Each subject completed half the items in each format. Results indicated little difference between forms, nor did subjects indicate a format preference. (Author/DWH)