Metacognition is important for successful goal-directed behavior. It consists of two main elements: metacognitive knowledge and online awareness. Online awareness consists of monitoring and self-regulation. Metacognitive sensitivity is the extent to which someone can accurately distinguish their own correct from incorrect responses and is an…

First impressions can influence rater-based judgments but their contribution to rater bias is unclear. Research suggests raters can overcome first impressions in experimental exam contexts with explicit first impressions, but these findings may not generalize to a workplace context with implicit first impressions. The study had two aims. First, to…

In many large-scale testing programs, examinees are frequently categorized into different performance levels. These classifications are then used to make high-stakes decisions about examinees in contexts such as in licensure, certification, and educational assessments. Numerous approaches to estimating the consistency and accuracy of this…

This dissertation examined classification outcome differences among four popular individual supervised machine learning (ISML) models (logistic regression, decision tree, support vector machine, and multilayer perceptron) when predicting minor class membership within imbalanced datasets. The study context and the theoretical population sampled…

The Accreditation Board for Engineering and Technology (ABET) which accredits higher education programs globally, has placed the ability for students to recognize professional and ethical responsibilities among their seven stated student outcomes that undergraduate engineering programs must ensure their students achieve to maintain their…

Background: Students with Intellectual Disability undergo frequent cognitive testing. Testing with this population is limited by insensitivity to relative strengths and weaknesses due to floor effects. Aim: The study explored the utility of deviation scores via four case studies as a supplement to educational decision-making. Methods: Four…

Comparative Judgement (CJ) is traditionally and primarily used to compare written texts. In this study we explored whether we could extend its use to comparing audio files. We used GCSE Music portfolios which contained a mix of audio recordings, musical scores and text documents. Fifteen judges completed two exercises: one comparing musical…

Consistency and coverage are two core parameters of model fit used by configurational comparative methods (CCMs) of causal inference. Among causal models that perform equally well in other respects (e.g., robustness or compliance with background theories), those with higher consistency and coverage are typically considered preferable. Finding the…

In a targeted double-scoring procedure for performance assessments that are used for licensure and certification purposes, a subset of responses receives an independent second rating if the first rating falls into a preidentified critical score range (CSR) where an additional rating would lead to considerably more reliable pass-fail decisions.…

Obtaining values for latent variables in factor analysis models, also referred to as factor scores, has long been of interest to researchers. However, many treatments of factor analysis do not focus on inference about the latent variables, and even fewer do so from a Bayesian perspective. Researchers may therefore be ill-acquainted with Bayesian…

This study assesses the technical efficiency of higher education institutions in terms of labor market outcomes for recent graduates, employing a comparative analysis of three distinct methodological approaches. Using a sample of recent graduates of Colombian universities who earned their degrees between 2007 and 2011, we estimated the…

Test scores are often used to make decisions about examinees, such as in licensure and certification testing, as well as in many educational contexts. In some cases, these decisions are based upon compensatory scores, such as those from multiple sections or components of an exam. Classification accuracy and classification consistency are two…

This research was carried out to analyze the efficiency of research in Indonesian universities and explore performance differences between knowledge areas. Data Envelopment Analysis (DEA) was used to evaluate the efficiency based on the data of 47 universities in the "Mandiri" cluster between 2014 and 2018 and 11 research fields…

High dropout rate exists universally in massive open online courses (MOOCs) due to the separation of teachers and learners in space and time. Dropout prediction using the machine learning method is an extremely important prerequisite to identify potential at-risk learners to improve learning. It has attracted much attention and there have emerged…

In the context of policy-driven language testing for citizenship, a growing body of research examines the political justifications and ethical implications of language requirements and test use. However, virtually no studies have looked at the role that language testers play in the evolution of language requirements. Critical gaps remain in our…