In computer-based tests allowing revision and reviews, examinees' sequence of visits and answer changes to questions can be recorded. The variable-length revision log data introduce new complexities to the collected data but, at the same time, provide additional information on examinees' test-taking behavior, which can inform test development and…

Multiple-choice questions (MCQs) are widely used in digital learning systems, as they allow for automating the assessment process. However, owing to the increased digital literacy of students and the advent of social media platforms, MCQ tests are widely shared online, and teachers are continuously challenged to create new questions, which is an…

A common feature of many assessment plans is the use of multiple-choice questions. Although there are criticisms of multiple-choice questions, this assessment format is here to stay--multiple-choice questions are effective means of evaluation in large classes, central to many licensing and entry exams, used in most adaptive learning platforms, and…

It is imperative to collect validity evidence prior to interpreting and using test scores. During the process of collecting validity evidence, test developers should consider whether test scores are contaminated by sources of extraneous information. This is referred to as construct irrelevant variance, or the "degree to which test scores are…

My dissertation research explored responder behaviors (e.g., demonstrating response styles, carelessness, and possessing misconceptions) that compromise psychometric quality and impact the interpretation and use of assessment results. Identifying these behaviors can help researchers understand and minimize their potentially construct-irrelevant…

The measurement of test-taking strategies and practices, mostly studied through qualitative methods, has been an important aspect of language testing and assessment research. The current study examines the test-taking strategies of International English Language Testing System (IELTS) test-takers and reports the process of designing and validating…

This booklet is a conceptual introduction to item response theory (IRT), which many large-scale testing programs use for constructing and scoring their tests. Although IRT is essentially mathematical, the approach here is nonmathematical, in order to serve as an introduction on the topic for people who want to understand why IRT is used and what…

Pedagogical content knowledge (PCK) is an important target of science teacher knowledge assessment. Most studies that have assessed the PCK across a large sample of science teachers used a text-based approach to elicit and assess the more declarative and static form of teachers' PCK. Recently, small-scale qualitative studies have adopted a novel…

Test-taking strategies are discussed in the literature as an important factor affecting test scores and are recommended to be taken into consideration regarding the validity of tests. Although studies have been conducted for more than a quarter century, no agreement has been reached on the dimensions of test-taking strategies. The purpose of this…

In previous studies, problem solving appraisal has been identified as playing a key role in promoting positive psychological well-being. The Problem Solving Inventory is the most widely used measure of problem solving appraisal and consists of 32 items. The length of the instrument, however, may limit its applicability to large-scale surveys…

Vocabulary tests administered on multilingual populations should take into account the unique linguistic and cultural makeup of the population by adopting test development methods that allow responses in several languages. Our aims were to develop a picture-naming test for multilingual Lebanese school-age children (L1: Lebanese, L2: French and/or…

This study aimed to investigate the test-taking strategies needed for successful completion of a lecture-based listening test by employing self-reported test-taking strategy use, actual strategy use measured via eye-tracking, and test scores. In this study, participants' gaze behavior (measured by fixation and visit duration and frequency) were…

This study examined the effects of bottom-up instruction (BI) on a high-stakes listening test for second language (L2) listeners and compared these effects with those of strategy instruction (SI) in relation to their proficiency levels. Two intact classes (48 L2 listeners) of different L2 listening proficiency levels received five weeks of BI and…

The extensive use of automated speech scoring in large-scale speaking assessment can be revolutionary not only to test design and rating, but also to the learning and instruction of speaking based on how students and teachers perceive and react to this technology. However, its washback remained underexplored. This mixed-method study aimed to…

As part of continued efforts to ensure inclusivity in assessment, OCR has developed a set of accessibility principles for question design in GCSE Science examinations, which has been applied since 2018. The principles are intended to help ensure that all students can demonstrate their knowledge, understanding and skills to the best of their…