Combining methods from earlier content validity and more contemporary content alignment studies may allow a more complete evaluation of the meaning of test scores than if either set of methods is used on its own. This article distinguishes item relevance indices in the content validity literature from test representativeness indices in the…

Priming examinees with questions about intended effort prior to testing has been shown to significantly increase examinee expended effort via self-reported effort and response-time effort. However, this question-behavior effect seems to wear off later in a testing session. We examined whether administering a second "dose" of the…

Pan and Wollack (PW) proposed a machine learning method to detect compromised items. We extend the work of PW to an approach detecting compromised items and examinees with item preknowledge simultaneously and draw on ideas in ensemble learning to relax several limitations in the work of PW. The suggested approach also provides a confidence score,…

We tested the impacts of Hofstede's cultural factors and mode of administration on item nonresponse (INR) for political questions in the European Values Study (EVS). We worked with the integrated European Values Study dataset, using descriptive analysis and multilevel binary logistic regression models. We concluded that (1) modes of administration…

In many countries, standardized math tests are important for achieving academic success. Here, we examine whether content of items, the story that explains a mathematical question, biases performance of low-SES students. In a large-scale cohort study of Trends in International Mathematics and Science Studies (TIMSS)--including data from 58…

Online tests offer many advantages but they still belong to assessment, which may make learners have anxiety. Thus, students may experience certain emotion. Academic emotion is a branch of emotion and has great effects on student learning. Such effects can be associated with individual differences, especially prior knowledge. To this end, this…

Formative assessments are primarily used as a tool to gauge learning throughout an anatomy course. They have also been demonstrated to improve student mastery and exam performance, although the precise nature of this relationship is poorly understood. In this study, it is hypothesized that formative assessment questions targeting higher cognitive…

In this edition of Assessment News, Charles Darr uses a case study from the National Monitoring Study of Student Achievement (NMSSA) to highlight how students can benefit from activating deliberate thinking in assessments. The key message underscores the value of using assessment information to generate insights that inform better teaching and…

The article offers an in-depth lexical analysis of the listening sub-test of the famous International English Language Testing System (IELTS). The vocabulary profile of 239 listening transcripts from 60 IELTS official practice tests were analyzed. The results showed that, 3,000 most frequent word families in the British National Corpus/Corpus of…

Performance on standardized academic aptitude tests (AAT) can determine important life outcomes. However, it is not clear whether and which aspects of the content of test questions affect performance. We examined the effect of psychological distance embedded in test questions. In Study 1 (N = 41,209), we classified the content of existing AAT…

Long-running surveys need a systematic way to reflect social change and to keep items relevant to respondents, especially when they ask about controversial subjects, or they threaten the items' validity. We propose a protocol for updating measures that preserves content and construct validity. First, substantive experts articulate the current and…

Educators have recently highlighted the importance of social science courses for students entering the medical field. This has led to the inclusion of sociological theories and concepts on the Medical College Admission Test (MCAT), a requirement for any student seeking formal medical training. Using open-ended survey data responses provided by…

For decades, the transmission of Cantonese opera in Hong Kong and China has faced considerable challenges, including a lack of valid assessment. This study aims to propose a holistic theoretical framework for the assessment of Cantonese operatic singing after {1) analysing two graded examinations of Peking opera and Cantonese opera and {2)…

Although the TOEFL iBT Listening test is sometimes used for other purposes, it was designed primarily for use as a college entrance examination. Item difficulty in TOEFL iBT Listening tests is the product of interactions between two sets of complex relationships: 1) relationships among numerous item characteristics themselves, and 2) relationships…

This article describes the content validation of the Transitioning Learners to Calculus in Community Colleges Institutional Self-Assessment Tool. The instrument comprises five content areas, each with an associated set of items representing practices to promote the success of underrepresented racially minoritized (URM) students as they transition…