ERIC Number: EJ1456535
Record Type: Journal
Publication Date: 2025
Pages: 20
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0022-0973
EISSN: EISSN-1940-0683
The Impacts of Small Teacher-Level Sample Sizes in Cluster-Randomized Trials
Journal of Experimental Education, v93 n1 p218-237 2025
Cluster-randomized trials have been widely used to evaluate the treatment effects of interventions on student outcomes. When interventions are implemented by teachers, researchers need to account for the nested structure in schools (i.e., students are nested within teachers nested within schools). Schools usually have a very limited number of teachers in each grade. This limited teacher-level sample size may pose potential challenges for study design, analysis, and reporting. The current study aims to evaluate the impacts of small teacher-level sample sizes on study design, analysis, and reporting in cluster-randomized trials. Through a simulation study, we show that the intraclass correlation coefficients at the teacher and school levels are prone to biased estimates in many cases, especially when the student-level sample size is also small. One design strategy we recommend is to sample a large number of students per teacher as this will alleviate the impacts of small teacher-level sample sizes. We recommend that researchers use the total variance, if available, rather than school-level variance to report standardized treatment effects for a meta-analysis. When the total variance is not available for all studies, researchers should use a school-level variance in two-level models as the base for standardization to reduce biases.
Descriptors: Sample Size, Multivariate Analysis, Randomized Controlled Trials, Correlation, Research Design, Statistical Analysis, Simulation, Statistical Bias, Educational Research
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A