Assessments are common in undergraduate classrooms, with formats including multiple-choice and open-ended (in which the students must generate their own answers) questions. While much is known about the strategies that students use when taking multiple-choice questions, there has yet to be a study evaluating the strategies that students employ…

In education, examining students' learning in detail, determining their strengths and weaknesses and giving effective feedback have gained importance over time. The aim of this study is to determine the distribution of students' answers to the reading comprehension achievement test items which were written at different cognitive levels and to…

As part of continued efforts to ensure inclusivity in assessment, OCR has developed a set of accessibility principles for question design in GCSE Science examinations, which has been applied since 2018. The principles are intended to help ensure that all students can demonstrate their knowledge, understanding and skills to the best of their…

Graded Response Method (GRM) is an alternative to multiple-choice testing where students rank options according to their relevance to the question. GRM requires discrimination and inference between statements and is a cost-effective critical thinking assessment in large courses where open-ended answers are not feasible. This study examined…

For test scores that rely on the accurate estimation of ability via an IRT model, their use and interpretation is dependent upon the assumption that the IRT model fits the data. Examinees who do not put forth full effort in answering test questions, have prior knowledge of test content, or do not approach a test with the intent of answering…

Creative question is generally considered as a tool to measure students' various levels of learning. The study focused on exploring the present situation of General Science test items/creative questions in Bangladesh. This descriptive study was conducted using a concurrent triangulation research design. To conduct this study both quantitative and…

Evidence to support the credibility of standard setting procedures is a critical part of the validity argument for decisions made based on tests that are used for classification. One area in which there has been limited empirical study is the impact of standard setting judge selection on the resulting cut score. One important issue related to…

Summative evaluation for large classes of first- and second-year undergraduate courses often involves the use of multiple choice question (MCQ) exams in order to provide timely feedback. Several versions of those exams are often prepared via computer-based question scrambling in an effort to deter cheating. An important parameter to consider when…

Background: A potential problem of low-stakes large-scale assessments such as the Programme for the International Assessment of Adult Competencies (PIAAC) is low test-taking engagement. The present study pursued two goals in order to better understand conditioning factors of test-taking disengagement: First, a model-based approach was used to…

Sometimes, test-takers may not be able to attempt all items to the best of their ability (with full effort) due to personal factors (e.g., low motivation) or testing conditions (e.g., time limit), resulting in poor performances on certain items, especially those located toward the end of a test. Standard item response theory (IRT) models fail to…

Chemistry faculty members are highly skilled in obtaining, analyzing, and interpreting physical measurements, but often they are less skilled in measuring student learning. This work provides guidance for chemistry faculty from the research literature on multiple-choice item development in chemistry. Areas covered include content, stem, and…

Test-preparation classes that focus on skills will benefit students beyond the test by developing skills they can use at university. This article discusses the purposes of various tests and outlines how to design effective test-prep classes. Several practical activities are included, and an appendix provides information on common standardized…

Through a project funded by the National Science Foundation, Horizon Research has been developing assessment items for students (in the process, compiling item-writing principles from several sources and adding their own). In this article, the authors share what they have learned about writing items that reveal student understanding, including…

According to the recent Nation's Report Card, 12th-graders failed to produce gains on the 2005 National Assessment of Educational Progress (NAEP) despite earning better grades on average. One possible explanation is that 12th-graders were not motivated taking the NAEP, which is a low-stakes test. We develop three Bayesian IRT mixture models to…

Over the last decade, high-stakes test preparation has crept into the inventory of developmentally responsive middle level instructional practices. Amid calls for increased accountability and more rigorous curriculum and academic standards, the middle school movement now finds itself in a spotlight of intense scrutiny. This article examines the…