Rater comments tend to be qualitatively analyzed to indicate raters' application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The…

Purpose: This paper aims to show how an Academic Literacies lens can contribute to a deeper understanding of writing for a professional doctorate (PD) by focusing both on the language of supervisors' written feedback and on student and supervisor perspectives on feedback throughout Year 1 (Y1). Design/methodology/approach: Firstly, written…

Generative artificial intelligence technologies, such as ChatGPT, bring an unprecedented change in education by leveraging the power of natural language processing and machine learning. Employing ChatGPT to assist with marking written assessment presents multiple advantages including scalability, improved consistency, eliminating biases associated…

Within the context of Chinese university education, effective communication in the field of second language writing heavily relies on lexical complexity, yet the role of writing feedback perception in relation to lexical complexity remains elusive. This study introduces a comprehensive writing feedback perception model encompassing perceptions of…

This study mainly explored the effects of teacher feedback, peer feedback and automated feedback on the use of metacognitive strategies in EFL writing. Ninety-seven participants were recruited and divided into three groups, who received two months of feedback from teachers, peers and an automatic writing evaluation system, respectively, and then…

In the existing literature, scholars have proposed various indices to measure the lexical richness (LR) of English as a foreign language (EFL) writing. However, there are currently issues of redundant indices and inconsistent usage. Attempting to address the research question of which indices are the most sensitive and effective ones to…

Responding to growing interests in alternative assessment practices, this article examines ungrading in two composition courses at a public university classified as a Hispanic-Serving Institution, detailing its implementation by two instructors in an Accelerated Learning Program composition class and a standalone composition class. Utilizing…

While the importance of feedback in education is well established, the effects of structured knowledge in collaborative academic writing remains uncertain. This study introduces an academic writing feedback tool that combines structured knowledge mining, analysis, and visualization. An empirical experiment was conducted in a second-year university…

We know little to what extent peer feedback strategies can be applied on a large scale in higher education for complex tasks. This study aimed to design, implement, and evaluate an online-supported peer feedback module for large-scale use to enhance higher education students' argumentative essay writing performance. To do this, 330 students from…

This quasi-experimental research investigates the employment of a formative assessment platform aided by artificial intelligence in an English public speaking course. The platform integrates deep learning, automatic speech recognition, and automatic writing evaluation. It provides automated assessment and immediate feedback on speakers' public…

The present study performs a three-level meta-analysis to investigate the overall effectiveness of automated writing evaluation (AWE) on EFL/ESL student writing performance. 24 primary studies representing 85 between-group effect sizes and 34 studies representing 178 within-group effect sizes found from 1993 to 2021 were separately meta-analyzed.…

Debate about the defining features of 'doctorateness' or what examiners look for in a PhD thesis is on-going, but hardly any attention has been paid in related literature to Postgraduate Researchers' (PGRs) perceptions of the key attributes that make a PhD thesis excellent. In this study we examine the state of play of the former to then go on to…

There is a current debate about the extent to which ChatGPT, a natural language AI chatbot, can disrupt processes in higher education settings. The chatbot is capable of not only answering queries in a human-like way within seconds but can also provide long tracts of texts which can be in the form of essays, emails, and coding. In this study, in…

This study explores how assessment criteria are applied in grading student work. It is found that explicit assessment criteria do not work as authoritative guidance as expected and that tacit criteria are more decisive in awarding a certain grade. Various sources that form idiosyncratic tacit criteria are identified. These sources, including…

Platform-enabled research services will control, manage, and measure learner experiences within that platform. In this paper, we consider the need for research services that examine learner experiences "outside" the platform. For example, we describe an effort to conduct an experiment on peer assessment in a college writing course, where…