ERIC Number: ED659699
Record Type: Non-Journal
Publication Date: 2023-Sep-27
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Professional Development in Reading Comprehension: A Meta-Analysis of the Effects on Teacher Practice and Student Outcomes
Marianne Rice; Kay Wijekumar; Kacee Lambright
Society for Research on Educational Effectiveness
Background/Context: Improving reading comprehension has been a persistent and perplexing dilemma around the world since written communication was invented. Reading comprehension, which is the goal of reading, is a multi-faceted construct intertwining code-focused early reading skills and meaning-focused comprehension skills and strategies (Connor et al., 2014; Scarborough 2001). A large body of research exists on code-focused reading instruction and interventions (National Reading Panel, 2000), teacher professional development (PD) on code-focused instruction (Hudson et al., 2021), and the effects on student outcomes. Despite the expectations of improved reading outcomes after strong early reading skills training, standardized reading comprehension assessments are persistently resistant to improvement (Catts, 2022). Research about comprehension shows that as children move through elementary school their early reading skills become automatized and thus make a smaller contribution to comprehension outcomes (Kim et al., 2010). Researchers and practitioners also agree that comprehension is best served when both code-focused and meaning-focused instruction are promoted in parallel (Gersten et al., 2001; Scarborough 2001). Consequently, equal emphasis and investments in research need to be placed on meaning-focused comprehension strategy and skill instruction regarding text processing that promote effective and efficient memory representations of text (Kintsch & van Dijk, 1978; van den Broek et al., 2005; Wijekumar et al., 2019). The NRP (2000) review found 203 studies to support teaching comprehension strategies and skills. However, only four studies were reviewed which focused on how to prepare teachers to implement comprehension instruction, suggesting more research in this area was critically needed. Thus, a new search identifying studies on reading comprehension PD and the potential impacts on student reading comprehension outcomes has the potential to influence policy and practice. Research Questions: (1) What is the summary effect of reading comprehension PD on teacher knowledge, teacher self-reported practice, and observed teacher practice outcomes?; (2) What is the summary effect of reading comprehension PD on student outcomes in reading comprehension?; (3) What are the characteristics of study design, participants, and PD provided that may potentially moderate these effects? Methods: A comprehensive literature search was conducted using an electronic database search, search of relevant prior reviews, and both a reference and forward search. A total of 1,448 results were identified and screened through the database search and an additional ten publications were identified through other search methods. A total of 47 publications met the inclusion and exclusion criteria. Criteria included: (1) Teachers were actively teaching in a kindergarten through grade 12 setting. (2) PD had a reading comprehension focus, which could include vocabulary and content area instruction. PD on code-focused reading skills was excluded. (3) The study had a group experimental design with a control group. (4) The study included a teacher outcome measure (i.e., teacher knowledge assessment, teacher self-reported practice, or observation of teacher practice) or a student outcome measure (i.e., reading comprehension assessment). (5) The study was written in English, but the language of instruction was not limited. After full-text review of the 47 publications, 38 were included in the meta-analysis. Study coding was focused on eight categorical explanatory moderators (study design, control group, grade level, coaching, PD format, content, PD provider, and type of outcome) and four continuous moderators (mean years of experience, advanced degree percentage, duration of PD, and number of comprehension skills covered). Standardized mean differences between the treatment and control groups were calculated using Hedges' g, which was selected because it corrects for small sample bias. Separate weighted, random-effect meta-regression was conducted for each potential moderator in order to estimate the effect on each of the summary weighted mean effect sizes and account for any variation in the overall teacher or student effect. Three methods were used to statistically assess potential publication bias in the corpus of identified publications: Egger's regression test, Vevea and Hedges' likelihood ratio test, and funnel plot symmetry with the trim- and-fill method. Results: Results of the random-effects RVE analysis on teacher outcomes showed, on average, teachers who had received PD on reading comprehension instruction outperformed the control group (i.e., teachers had not received the PD) on outcomes related to teacher knowledge or practice of reading comprehension instruction. A statistically significant, large weighted random mean effect size was calculated for the overall main effect, g = 0.947 (SE = 0.125, 95% CI [0.688, 1.21], p < 0.001). The variability in effect sizes estimated to be true effect size heterogeneity rather than sampling error was large (I[superscript 2] = 86.21%, [tau][superscript 2] = 0.38). Results of the random-effects RVE analysis on student outcomes showed, on average, students whose teachers had received PD on reading comprehension instruction outperformed the control group (i.e., students whose teachers had not received the PD) on reading comprehension outcomes. The small, statistically significant weighted random mean effect size was g = 0.248 (SE = 0.069, 95% CI [0.107, 0.388], p < 0.01). A large proportion of the variability in effect sizes was estimated to be true effect size heterogeneity rather than sampling error (I[superscript 2]= 95.79%, [tau][superscript 2] = 0.07), suggesting moderation analyses would be appropriate to explore further. No statistically significant differences were found for any of the moderators. Discussion: Our meta-analysis examined whether teacher participation in reading comprehension PD had an overall effect on teacher knowledge and practice, and student outcomes. Results indicate that teachers who received the PD performed better on measures of teacher knowledge and use of practices than teachers who did not receive the PD. Additionally, students of teachers who had received the PD had better performance on reading comprehension measures than students in classes of teachers who did not experience the PD. Limitations for our study include the lack of standardized measures for teacher outcomes in the studies, and the small number of studies may have had limited power to detect differences for the moderator analyses. More research is needed carefully examining the specific components that make teacher PD effective and for which teachers.
Descriptors: Elementary School Students, Elementary School Teachers, Reading Comprehension, Critical Reading, Reading Instruction, Professional Development, Inservice Teacher Education, Reading Achievement
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research; Information Analyses
Education Level: Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A