The Lognormal Response Time (LNRT) model measures the speed of test-takers relative to the normative time demands of items on a test. The resulting speed parameters and model residuals are often analyzed for evidence of anomalous test-taking behavior associated with fast and poorly fitting response time patterns. Extending this model, we…

When tests are low stakes for examinees, meaning there are little to no personal consequences associated with test results, some examinees put little effort into their performance. To understand the causes and consequences of diminished effort, researchers correlate test-taking effort with other variables, such as test-taking emotions and test…

In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of…

Considerable research has examined the use of rapid guessing measures to identify disengaged item responses. However, little is known about students who rapidly guess over the course of several tests. In this study, we use achievement test data from six administrations over three years to investigate whether rapid guessing is a stable trait-like…

This paper explores the impact of test-taking conditions on the quality of the Programme for the International Assessment of Adult Competencies (PIAAC) assessment. Interviewers record information about the room of assessment and interruptions that occurred during each interview. These observations, along with information on interviewer assignment…

This classroom experiment investigates the effects of adding representational pictures to multiple-choice and constructed-response test items to understand the role of the response format for the multimedia effect in testing. Participants were 575 fifth- and sixth-graders who answered 28 science test items--seven items in each of four experimental…

Word Associates Format (WAF) tests are often used to measure second language learners' vocabulary depth with a focus on their network knowledge. Yet, there were often many variations in the specific forms of the tests and the ways they were used, which tended to have an impact on learners' response behaviors and, more importantly, the psychometric…

The overall purpose of the research studies described in this report was to investigate the quality of the administration of the Partnership for Assessment of Readiness for College and Careers (PARCC) assessment during its first operational year (2014-2015). Findings from these studies can be used to identify threats to the validity of PARCC test…

Sometimes, test-takers may not be able to attempt all items to the best of their ability (with full effort) due to personal factors (e.g., low motivation) or testing conditions (e.g., time limit), resulting in poor performances on certain items, especially those located toward the end of a test. Standard item response theory (IRT) models fail to…

Questionnaire instruments are frequently administered in digital formats, largely web-based, without much systematic investigation of possible effects from these administration methods. Furthermore, little attention has been given to the contextual lack of control for extraneous factors that may influence user responses. In this study, 263…

This is an account of a portion of the research on cognitive, personality, and social psychology at ETS since the organization's inception. The topics in cognitive psychology are the structure of abilities; in personality psychology, response styles and social and emotional intelligence; and in social psychology, prosocial behavior and stereotype…

The linear logistic test model (LLTM) breaks down the item parameter of the Rasch model as a linear combination of some hypothesized elementary parameters. Although the original purpose of applying the LLTM was primarily to generate test items with specified item difficulty, there are still many other potential applications, which may be of use…

Questionnaire instruments are routinely translated to digital administration systems; however, few studies have compared the differential effects of these administrative methods, and fewer yet in authentic contexts-of-use. In this study, 326 university students were randomly assigned to one of two administration conditions, paper-based (PBA) or…

Compared Edwards Personality Inventory (EPI) reliability under the Edwards' instructions (subject responds to the items as he believes those who know him best would answer if asked to describe him) versus the more conventional instructions (to describe yourself directly as you see yourself). (Author/RC)