This study aims to examine the potential differences between teacher evaluations and artificial intelligence (AI) tool-based assessment systems in university examinations. The research has evaluated a wide spectrum of exams including numerical and verbal course exams, exams with different assessment styles (project, test exam, traditional exam),…

The Music Performance Self-Efficacy Scale (MPSES) is an important scale designed to reflect the four sources of self-efficacy of Bandura by Zelenak, and has been used in many studies of music education in the international literature in recent years. This study was carried out to ensure the validity and reliability of the Turkish translation of…

This article focuses on rater severity and consistency and their relation to different types of rater experience over a long period of time. The article is based on longitudinal data collected from 2009 to 2019 from the second language Finnish speaking subtest in the National Certificates of Language Proficiency in Finland. The study investigated…

A major challenge to promoting effective early science and engineering education is the lack of reliable and validated assessments that align with current educational guidelines for science and engineering. Existing early science and engineering assessments either cover a narrow range of concepts and practices and/or are not designed in a way to…

Background: A growing number of surveys are conducted online where respondents can choose to complete the questionnaire (Lehdonvirta et al., 2020). As respondents are self-selected, there is potential that the respondents will not be an accurate representation of the population. For example, white people are disproportionately more likely to…

Purpose: This study compares two electromagnetic articulographs manufactured by Northern Digital, Inc.: the NDI Wave System (from 2008) and the NDI Vox-EMA System (from 2020). Method: Four experiments were completed: (1) comparison of statically positioned sensors; (2) tracking dynamic movements of sensors manipulated using a motor-driven LEGO…

We have limited understanding of why reviewers tend to strongly disagree when scoring the same research proposal. Thus far, research that explored disagreement has focused on the characteristics of the proposal or the applicants, while ignoring the characteristics of the reviewers themselves. This article aims to address this gap by exploring…

Buros' "Mental Measurements Yearbook (MMY)" has provided professional reviews of commercially published psychological and educational tests for over 80 years. It serves as a kind of conscience for the testing industry. For a random sample of 50 entries in the "19th MMY" (a total of 100 separate reviews) this study determined…

The reliability of information in the global information space is one of the most important problems of globalization. The credibility of various information resources is currently being studied and considered in different ways. In some cases, the problem of the reliability of information can be assessed as harmful and dangerous. This article,…

The validity of the Kappa coefficient of chance-corrected agreement has been questioned when the prevalence of specific rating scale categories is low and agreement between raters is high. The researchers proposed the Lambda Coefficient of Rater-Mediated Agreement as an alternative to Kappa to address these concerns. Lambda corrects for chance…

In response to the call for more rigorously validated educational assessments, this study used an iterative multimethod validation process to develop and validate outcomes from the 21st Century Skills Assessment global rating scale. Qualitative and quantitative data sources were used to inform four types of validity evidence: content, response…

Several school districts in Georgia use two teacher-administered diagnostic assessments of student knowledge of mathematics as part of their multi-tiered system of support in grades K-8: the Global Strategy Stage (GloSS; New Zealand Ministry of Education, 2012) and the Individual Knowledge Assessment of Number (IKAN; New Zealand Ministry of…

This document are the appendixes for the report, "The Reliability and Consequential Validity of Two Teacher-Administered Student Mathematics Diagnostic Assessments." Rather than relying on occasional testimonials from the field, decisions about using diagnostic assessments across the state should be based on psychometric data from an…

Teachers need to assess their students' current level of mathematical understanding to provide appropriate interventions for students who are struggling. Several school districts in Georgia currently use two assessments for this purpose--the Global Strategy Stage (GloSS) and the Individual Knowledge Assessment of Number (IKAN). The IKAN is…

Conventional methods for evaluating the utility of subscores rely on traditional indices of reliability and on correlations among subscores. One limitation of correlational methods is that they do not explicitly consider variation in subtest means. An exception is an index of score profile reliability designated as [G], which quantifies the ratio…