The Research Excellence Framework is a high-stakes exercise used by the UK government to allocate billions of pounds of quality-related research (QR) funding and used by the media to rank universities and their departments in national league tables. The 2008, 2014 and 2021 assessments were zero-sum games in terms of league table position because…

Indirect questioning attempts to overcome social desirability bias in survey research. However, to properly analyze the resulting data, it is crucial to understand how it impacts responses. This study analyzes results from a randomized experiment that tests whether direct versus indirect questioning methods lead to different results in a sample of…

The aim of this research was to analyze and compare analytic rubric and general impression scoring in peer assessment. A total of 66 university students participated in the study and six of them were chosen as peer raters on a voluntary basis. In the research, students were supposed to prepare a sample study within the scope of scientific research…

Due to myriad applications of the nominal group technique (NGT), a highly flexible iterative focus group method, researchers know little about its optimal scoring procedures. Exploring benefits and biases that such procedures might present, we aim to clarify how NGT scoring systems can privilege consensus or prioritization. In conducting the first…

The present study aimed to determine whether using culture-themed documentaries in Turkish teaching of B2 level international students has effects on the development of guided writing skills. The study had an action-research design, which is one of the qualitative study approaches. The study group consisted of 12 international students at the B2…

The advent of artificial intelligence (AI), particularly large language models (LLMs) like ChatGPT and Gemini, has significantly impacted the educational landscape, offering unique opportunities for learning and assessment. In the realm of written assessment grading, traditionally viewed as a laborious and subjective process, this study sought to…

The aim of this paper is to discuss the role of coding and observation manuals in classroom studies. While observation manuals have been a part of the methodological toolkit for measuring various aspects of instruction for decades, the field has also been suffering from "paradigm wars," fragmentation and local production of instruments.…

The field of multilingual testing and assessment has grown rapidly in recent years due to the widespread need to integrate immigrant populations into mainstream education and to provide fair and equitable forms of assessment for all students. However, a continuing emphasis on bilingual students has created a significant gap in testing and…

In this special issue on high-stakes English language testing in China, the two articles on computer-based testing (Jin & Yan; He & Min) highlight a number of consistent, ongoing challenges and concerns in the development and implementation of the nationwide IB-CET (Internet Based College English Test) and institutional computer-adaptive…

Although International Association for the Evaluation of Educational Achievement-pioneered international large-scale assessment (ILSA) of education is now a well-established science, non-practitioners and many users often substantially misunderstand how large-scale assessments are conducted, what questions and challenges they are designed to…

In this article we describe a three-year study that was conducted in three phases to evaluate the feasibility of assessing digitized portfolios of student creative work for high-stakes purposes. The first two phases suggested that creative work could be digitized with adequate fidelity, and that students could submit their own work from schools to…

Computer games are effective instructional tools used in programming courses to increase students' motivation and engagement. This participatory action research aims to redesign the Object-Oriented Programming course in which the first author is both the instructor and researcher to make it more effective and efficient. In the first step of the…

This paper provides insights in the development of a skill matching test which addresses soft skills integrated videos as media to provide information about situations to be rated. The design of the skill testing and matching tool is situated in the educational ERASMUS+ project SMART which is presented as well. With a specific view on team work…

Scientific epistemology is important issues in the International science education, of which teachers' views of nature of science and scientific inquiry as the two core topics, become the principal part of research, also have an important influence on the theory and practice of science curriculum. The purpose of this study was to investigate and…

Previous research in second language writing has shown that when scoring performance assessments even trained raters can exhibit significant differences in severity. When raters disagree, using discussion to try to reach a consensus is one popular form of score resolution, particularly in contexts with limited resources, as it does not require…