The typical process for assessing inter-rater reliability is facilitated by training raters within a research team. Lacking is an understanding if inter-rater reliability scores "between" research teams demonstrate adequate reliability. This study examined inter-rater reliability between 16 researchers who assessed fundamental motor…

This article focuses on rater severity and consistency and their relation to major changes in the rating system in a high-stakes testing context. The study is based on longitudinal data collected from 2009 to 2019 from the second language (L2) Finnish speaking subtest in the National Certificates of Language Proficiency in Finland. We investigated…

There can be many reasons why students fail to answer correctly to summative tests in advanced computer science courses: often the cause is a lack of prerequisites or misconceptions about topics presented in previous courses. One of the ITiCSE 2020 working groups investigated the possibility of designing assessments suitable for differentiating…

This article focuses on rater severity and consistency and their relation to different types of rater experience over a long period of time. The article is based on longitudinal data collected from 2009 to 2019 from the second language Finnish speaking subtest in the National Certificates of Language Proficiency in Finland. The study investigated…

The aim of this study was to validate a teacher rating scale (TRS-EN) for assessing early numeracy skills and for identifying mathematically low-performing 3- and 4-year-old children. Fifty-six preschool teachers assessed children's (3-year-olds = 127; 4-year-olds = 145) numerical relational, counting, and basic arithmetic skills. Construct…

When classroom observations are increasingly used for accountability and evaluation purposes, a deeper understanding of the psychometric properties of such measurement tools is needed. The present study took a unique approach to examine the psychometric properties of a commonly used classroom observation measure by testing the reliability of…

This study investigates children's vocabulary knowledge in Finnish Sign Language (FinSL), specifically their understanding of different form-meaning mappings by using a multilayered assessment format originally developed for British Sign Language (BSL). The web-based BSL vocabulary test by Mann (2009) was adapted for FinSL following the steps…

This research examines psychometric properties of the 10-item Perceived Academic Social Influence Scale (PASIS) across gender groups with samples from Finland and Taiwan (Total N = 4727, 50.2% female adolescents). Results of composite scores, factor loadings, item-total correlations, and values of square root of Average Variance Extracted (AVE)…

This study belongs to assessment-related research and aimed to investigate Finnish high-school students' (n = 211) topic-specific epistemic beliefs about climate change and whether the Norwegian topic-specific epistemic beliefs questionnaire (TSEBQ) was also valid among Finnish respondents. Thus, research data were not only derived from the TSEBQ…

Attachment difficulties are associated with a range of adverse outcomes in mental health, and people with intellectual disabilities (IDs) may be at greater risk of experiencing difficulties in their attachment relationships. This review critically evaluated recent research measuring the prevalence of attachment difficulties in people with ID.…