The goal of this study is to determine the reliability of the performance points of dentistry students regarding communication skills and to examine the scoring reliability by generalizability theory in balanced random and fixed facet (mixed design) data, considering also the interactions of student, rater and duty. The study group of the research…

The purpose of the current study is to validate the Counseling Competencies Scale Revised (CCS-R). We used convenience sampling to recruit counselors and counselors-in-training (N = 156). Specifically, we had 130 female (83.3%) and 26 male (16.7%) participants, and the mean age was 33.16 (SD = 11.18), ranging from 20 to 62 years old. We used…

A Bayesian IRT-model approach was used to investigate the validity and reliability of student perceptions of teaching quality. Furthermore, the student perceptions were compared with ratings of teaching quality by external observers. Grade 4 students (n = 675) filled out a questionnaire that was used to measure their opinions about the lessons of…

This study examined the exchangeability of total scores (i.e., intelligent quotients [IQs]) from three brief intelligence tests. Tests were administered to 36 children with intellectual giftedness, scored live by one set of primary examiners and later scored by a secondary examiner. For each student, six IQs were calculated, and all 216 values…

Average score change: How big is big? This paper discusses school-level changes in average ACT scores and highlights an interactive tool designed to facilitate score change comparisons.

Learning geography in Indonesia philosophically aims to develop spatial literacy. Students must improve spatial literacy to form reasoning skills and apply spatial concepts in real life. Applying Gersmehl's spatial learning can improve students' spatial literacy through syntax arranged based on spatial aspects. The use of google earth helps…

In response to the absence of an instrument to measure educator inspiration with evidence of validity and reliability, the authors developed the "Educator Inspire Scale" (EIS), an assessment designed to assess the construct of inspiration in educators. Therefore, their investigation examined the psychometric properties of the EIS scores…

With the exponential growth of online information seeking by young people, it is imperative for health and sexual health educators to consider online information a resource young people will pursue. Access to accurate and comprehensive sexual health information is important, yet there is a scarcity of research evaluating the quality of this…

Both school and district administrators use the results of standardized, large-scale tests to inform decisions about the need for, or success of, educational programs and interventions. However, test results at the school level are subject to random fluctuations due to changes in cohort, test items, and other factors outside of the school's…

We evaluated the score stability of the Framework for Teaching (FFT), a prominent observation instrument used for teacher evaluation. Three raters each scored 200 reading and mathematics lessons taught by 20 kindergarten teachers. Using Generalizability theory analyses, we decomposed the FFT's Classroom Environment, Instruction, and Total scores…

We examined data from a national sample of 917 school counselors to determine the factor structure of the School Counselor STEM Advocacy Survey. An exploratory and confirmatory factor analysis supported use of the two-factor model. Survey scores demonstrated good internal consistency and convergent validity. We discuss differences between key…

The Many-Facet Rasch model is frequently used to analyse and minimize disparities in rater (judge) severity in performance evaluations, in which raters assign scores to test-takers' performances. In this research, the aim of the present study was to analyse science teacher candidates' laboratory activities by using the Many-facet Rasch model.…

Background & Aims: This study ran within the framework of the Thales Aphasia Project that investigated the efficacy of elaborated semantic feature analysis (ESFA). We evaluated the treatment integrity (TI) of ESFA, i.e., the degree to which therapists implemented treatment as intended by the treatment protocol, in two different formats:…

Many researchers use Cronbach's alpha to demonstrate internal consistency, even though it has been shown numerous times that Cronbach's alpha is not suitable for this. Because the intention of questionnaire and test constructers is to summarize the test by its overall sum score, we advocate summability, which we define as the proportion of total…

When cognitive and educational tests are administered under time limits, tests may become speeded and this may affect the reliability and validity of the resulting test scores. Prior research has shown that time limits may create or enlarge gender gaps in cognitive and academic testing. On average, women complete fewer items than men when a test…