A mixture of logit models is proposed that discriminates between responses to rating questions that are affected by a tendency to prefer middle or extremes of the scale regardless of the content of the item (response styles) and purely content-driven preferences. Explanatory variables are used to characterize the content-driven way of answering as…

Web, or online, probing has the potential to supplement existing questionnaire design processes by providing structured cognitive data on a wider sample than typical qualitative-only question evaluation methods can achieve. One of the practical impediments to the further integration of web probing is the concern of survey managers about how the…

Questionnaire scales that are mixed-worded, i.e. include both positively and negatively worded items, often suffer from issues like low reliability and more complex latent structures than intended. Part of the problem might be that some responders fail to respond consistently to the mixed-worded items. We investigated the prevalence and impact of…

Forced-choice questionnaires have been proposed to avoid common response biases typically associated with rating scale questionnaires. To overcome ipsativity issues of trait scores obtained from classical scoring approaches of forced-choice items, advanced methods from item response theory (IRT) such as the Thurstonian IRT model have been…

The effect of the response scale direction on response behavior is a well-known phenomenon in survey research. While there are several approaches to explaining how such response order effects occur, the literature reports mixed evidence. Furthermore, different question formats seem to vary in their susceptibility to these effects. We therefore…

This study investigates consistent and inconsistent respondents to mixed-worded questionnaire scales in large-scale assessments. Mixed-worded scales contain both positively and negatively worded items and are universally applied in different survey and content areas. Due to the changing wording, these scales require a more careful reading and…

Even when relatively infrequent, careless and random responding (C/RR) can have robust effects on individual and group data and thereby distort clinical evaluations and research outcomes. Given such potential adverse impacts and the broad use of self-report measures when appraising addictions and addictive behavior, the detection of C/RR can…

There has long been the suspicion amongst staff in Art & Design that the ratings given to their subject disciplines in the UK's National Student Survey are adversely affected by a combination of circumstances--a "perfect storm". The "perfect storm" proposition is tested by comparing ratings for Art & Design with those…

This article proposes procedures for simultaneously assessing and controlling acquiescence and social desirability in questionnaire items. The procedures are based on a semi-restricted factor-analytic tridimensional model, and can be used with binary, graded-response, or more continuous items. We discuss procedures for fitting the model (item…

A literature review and bibliography on questionnaire construction are presented. The broad definition of questionnaire includes scales, structured interview forms, survey forms, and similar paper and pencil instruments used to elicit responses and collect information. A comprehensive literature search of journal articles, books, and reports in…

The tendency of some respondents to omit items more often when they feel they have a less positive evaluation to make and less frequently when the evaluation is more positive is discussed. Five examples illustrate this form of nonresponse bias. Recommendations to overcome nonresponse bias are offered. (SLD)

The problem of response set is important for questionnaire designers and interpreters, but the public is affected as well if policy is determined on the basis of unsupported conclusions. This study focused on one of the most researched response sets, extreme responding (ER), or extreme checking styles, and its relationship to one dimension of…

Consistency of gain from changing test answers was tested for students instructed about answer-changing research results, and composition of the gain was analyzed by examining the students' reasons for changing. Mean gain remained positive and consistent with gain for previously studied uninstructed groups; amount of change was also stable.…

Content-parallel questionnaire items in which response schemes varied in one of two ways--scale alternatives were all defined or only endpoints were defined, and alternatives were numbered or lettered--were investigated on a large sample of college freshmen. (Author/JKS)

In a replication of an earlier study, a questionnaire with items lacking content and merely containing answer possibilities was administered to a sample of Dutch freshmen psychology students. Subjects showed a preference for positive options over negative options. (Author/JKS)