Questionnaires in educational research assessing students' attitudes and beliefs are low-stakes for the students. As a consequence, students might not always consistently respond to a questionnaire scale but instead provide more random response patterns with no clear link to items' contents. We study inter-individual differences in students'…

Formative assessments are an important component of massive open online courses (MOOCs), online courses with open access and unlimited student participation. Accurate conclusions on students' proficiency via formative, however, face several challenges: (a) students are typically allowed to make several attempts; and (b) student performance might…

In undergraduate research settings, students are likely to encounter anomalous data, that is, data that do not meet their expectations. Most of the research that directly or indirectly captures the role of anomalous data in research settings uses post-hoc reflective interviews or surveys. These data collection approaches focus on recall of past…

Inattentiveness and social desirability might be particularly problematic for self-report scales in creativity and arts research. Respondents who are inattentive or who present themselves favorably will score highly on scales that yield positively skewed distributions and that assess socially valued constructs, such as scales measuring creativity…

When dealing with missing responses, two types of omissions can be discerned: items can be skipped or not reached by the test taker. When the occurrence of these omissions is related to the proficiency process the missingness is nonignorable. The purpose of this article is to present a tree-based IRT framework for modeling responses and omissions…

The overall purpose of the research studies described in this report was to investigate the quality of the administration of the Partnership for Assessment of Readiness for College and Careers (PARCC) assessment during its first operational year (2014-2015). Findings from these studies can be used to identify threats to the validity of PARCC test…

Proper survey design is essential to obtain reliable, replicable data from research subjects. One threat to inferences drawn from surveys is anchoring-and-adjusting. Tversky and Kahnemann (1974) observed that participants' responses to questions depended systematically on irrelevant information they received prior to answering. It is important for…

The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method against the oral examination (OE) method. MCQs are widely used and their importance seems likely to grow, due to their inherent suitability for electronic assessment. However, MCQs are influenced by the tendency of examinees to guess…

The influence of scale format on results was examined. Two Likert type formats, one with all choice points defined and one with only end-points defined, were administered. Each subject completed half the items in each format. Results indicated little difference between forms, nor did subjects indicate a format preference. (Author/DWH)

Reasons for not returning a questionnaire were studied among Lutheran church members sent a postcard to indicate no intention of replying. The analysis is based on 431 returned postcards. For most nonresponders, the interest value of the questionnaire topic was not high enough. Many who returned the negative reply card would be resistant…

Item-response changing as a function of test anxiety was investigated. Data supported the hypothesis that high test-anxious students make more item-response changes than low test-anxious students. Also, both high- and low-anxious students profit to a similar extent proportionally from answer changing. (Author/GK)

Developed a scale for detecting random responses on the Multidimensional Pain Inventory using 95 undergraduates, 34 chronic pain patients, and 115 health-care professionals. A variable response scale was developed that discriminated accurately between valid and random profiles in two cross-validation samples, predicting random profiles with 90%…

The susceptibility to faking on the Tennessee Self Concept Scale was examined among college students. Additionally, groups of respondents, instructed to respond in a "random" fashion to pre-determined numbers of items in the TSCS, were subjected to a plausibility analysis of their test response vectors using the Rasch measurement model.…

Effects of item presentation mode on degree of leniency bias in responses to field research questionnaires were studied. Two modes were examined: first with items measuring the same dimensions grouped together and second with such items distributed randomly. The random mode showed substantially less leniency response bias. (Author/BW)

The empirical evidence does not support the prevailing conventional wisdom that it is advisable to mix positively and negatively worded items in psychological measures to counteract acquiescence response bias. An experiment, evaluating subjects' ability to respond accurately to both positive and reversed items on a questionnaire, analyzed post-hoc…