Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers' evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations…

The large scale expansion of online courses has led to the crisis of course quality issues. In this study, we first established an evaluation index system for online courses using factor analysis, encompassing three key constructs: course resource construction, course implementation, and teaching effectiveness. Subsequently, we employed factor…

In the existing literature, scholars have proposed various indices to measure the lexical richness (LR) of English as a foreign language (EFL) writing. However, there are currently issues of redundant indices and inconsistent usage. Attempting to address the research question of which indices are the most sensitive and effective ones to…

Assessment of learners in the anatomical sciences is a complex task as it not only tests students' fact knowledge, but also the analysis of visual information. Sometimes, novice histology learners must acquire image recognition skills of microscopic structures in a short time frame. This paper describes a strategy of offering first year dental…

This paper first analyzed the shortcomings of the summative final exam methods which focuses only on final exams. Then, we showed the specific implementation of the quantitative formative assessment which includes objective results and subjective scores. Objective results include the traditional written chapter test score, online mid-term and…

In this study, we estimated the relationship between two popular measures of teacher effectiveness--teachers' value-added model (VAM) estimates, represented in this study via median growth percentiles (MGPs), and teachers' observational scores, derived from the TAP System for Teacher and Student Advancement. We examined the relationship between…

Purpose: The OSCE is regarded as the gold standard of competence assessment in many healthcare programs, however, there are numerous internal and external sources of variation contributing to checklist marks. There is concern amongst organisers that candidates may be unfairly disadvantaged if they follow an "excellent" preceding…

The Many-Facet Rasch model is frequently used to analyse and minimize disparities in rater (judge) severity in performance evaluations, in which raters assign scores to test-takers' performances. In this research, the aim of the present study was to analyse science teacher candidates' laboratory activities by using the Many-facet Rasch model.…

In-class group work activities are found to promote the interpersonal skills of learners. To support the teachers in facilitating such activities, we designed a learning analytics-enhanced technology framework, Group Learning Orchestration Based on Evidence (GLOBE) using data-driven approaches. In this study, we implemented the algorithmic group…

Teacher preparation programs are increasingly expected to use data on preservice teacher (PST) skills to drive program improvement and provide targeted supports. Observational ratings are especially vital, but also prone to measurement issues. Scores may be influenced by factors unrelated to PSTs' instructional skills, including rater standards.…

This paper aims to identify and compare key factors influencing US States' or an OECD country's decision to employ student test scores or test score value-added (VAM) in evaluating teacher effectiveness. Analysis using multiple linear probability model and logistic regression frameworks indicates that the use of VAM/student test scores was…

The purpose of this multiple intrinsic case study was to describe how Northeastern United States middle school teachers and students engaged with a new automated writing evaluation tool used to score and provide feedback on extended essay assignments to improve teaching and learning writing. Richard Elmore's (1993) instructional core framework is…

LXD Research was engaged by 95 Percent Group to conduct a third-party, mixed-methods efficacy study of their 95 Phonics Core Program (95 PCP) to evaluate the impact on fourth and fifth grade literacy outcomes over the school year 2023-2024. LXD analyzed the formative assessment data from i-Ready, focusing on overall scale scores, subdomains, and…

Given the shift toward technology-based approaches to assessment and intervention and the growing popularity of virtual service delivery, practitioners must have data to inform whether student performance on curriculum-based measures (CBMs) is equivalent when administering assessments through different modalities. Reading CBMs are a widely used…

Calligraphy is the treasure of traditional Chinese culture and it is ubiquitous in China. Because of the unique aesthetic characteristics of calligraphy, the difficulty of calligraphy teaching lies in the assessment of calligraphy works. The major problems concerning calligraphy assessment are the low time efficiency of calligraphy assessment and…