ERIC Number: ED663574
Record Type: Non-Journal
Publication Date: 2024-Sep-21
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Supporting Students' Math Achievement with Adaptive and Game-Based EdTech: A Longitudinal Efficacy Study of ExploreLearning's Frax Program
Megan Conrad; David Shuster
Society for Research on Educational Effectiveness
Background: Performance with fractions has been a weak point in U.S. education for decades and has not improved in recent years (Siegler, 2017). High School math teachers frequently rate their students' knowledge of fractions as "poor" and see this lack of foundational knowledge as one of the top barriers to students mastering algebra (Hoffer et al., 2007). Prior research has found that fractions knowledge in 5th grade uniquely predicts mathematics achievement in high school, even after controlling for variables like general intellectual ability, working memory, and family income and education levels (Siegler et al., 2012). EdTech has the potential to deliver personalized classroom interventions at scale to improve fractions knowledge in a manner that is both efficient and effective. The current study tests the impact of Frax: a game-based, adaptive learning program that uses research-based instructional methods to support foundational fractions skills. Research Questions: A two-year longitudinal study (Fall 2021 through Spring 2023) with elementary school students was conducted to measure the impact of Frax usage on fractions knowledge and mathematics achievement. The study used a quasi-experimental methodology with case-control matching to create equivalent student groups at baseline. Student product usage and math performance were measured in collaboration with the school district. The study aimed to address the following research questions: (1) Does usage of the Frax program improve students' performance on standardized math assessments? (2) Do students who use the Frax program with fidelity show greater improvement on math assessments over time compared to matched, control students? (3) Do the effects of the Frax program vary for students of different ability levels? Participants: The sample (Table 1) included 2,440 3rd-grade and 2,620 4th-grade students from a large, suburban public school district in the southeastern United States. The district's minority enrollment is over 60% and 35% of students are economically disadvantaged. The sample consisted of 35% Hispanic/Latino and 15% Black/African American students. The majority of students, 79%, scored at least 1 grade level below standards on the baseline math test. Intervention: The Frax Foundations I program, broadly aligned to 3rd-grade fractions standards, contains 27 lessons (Missions) which take 30 minutes each to complete. Lessons are typically completed in class with teachers monitoring student progress to deliver real-time support. Frax is designed as a zero-entry program; students with no previous knowledge of fractions can begin using the program immediately. The Frax approach teaches students that fractions are numbers with magnitude like any other number. Students practice these concepts through explicit, scaffolded use of length models, number lines, and measurements. Students build the conceptual foundation they need to move on to fractions arithmetic. The adaptive algorithms behind Frax are designed to maximize student engagement and motivation, automatically recognizing and delivering the right level of support to students. This creates individualized, efficient instruction that intentionally moves each student toward mastery. Research Design: This study used a quasi-experimental, case-control matching design. All 3rd-5th grade teachers in the district had access to the Frax program and were eligible to receive professional development. Program usage was measured by the number of missions students completed each year. Baseline achievement was quantified by Fall 2021 i-Ready Diagnostic Assessment math scale scores and performance relative to grade-level proficiency (2+ grade levels below, 1-grade level below, on grade level or above). Matching was conducted in Spring 2022, creating matched pairs of Frax users and non-users on current grade level (3rd, 4th) and Fall 2021 i-Ready math scale scores (±5 points). Early growth (Spring 2022 and Fall 2022) was measured by the I-Ready Diagnostics assessment. In Spring 2023, the district moved to a state-specific diagnostic test. Overall math scale scores, fractions subscale scores, and relative achievement categories (below standards, at/near standards, and above standards) were used to determine the impact of Frax usage on student math growth. Analysis: SPSS was used for all analyses. Paired samples t-tests and standardized effect sizes (Hedge's g) were used to determine the magnitude of gains in test scores comparing across matched users and non-users. Categorical 2x2 chi-square tests were used to look at the likelihood of meeting grade level expectations for users and non-users. Findings/Results: Results from Year 1 found that fidelity program usage led to statistically significantly greater improvement in math compared to paired non-users. Compared to previous metanalyses (Lipsey et al., 2012), Frax was 3x more effective at improving math scores than the average educational intervention for 3rd graders and 5x more effective for 4th graders (Table 2). Frax usage was also related to students' achievement of grade level proficiency; 47% of high Frax users met grade level expectations in the spring compared to only 31% of students in the control group, X[superscript 2 ](1, N = 1008) = 4.65, p = 0.031. Importantly, this effect was observed for students within every student baseline achievement category (Figure 1). The results from year 2 found that the achievement gains associated with Frax usage in year 1 were sustained: in Spring 2023 high Frax users had significantly higher overall math scores, t(170) = 4.044, p < 0.001, and were significantly more likely to score above standards on the Fractions subscale, X[superscript 2] (1, N = 342) = 5.78, p = 0.016, compared to their matched non-users (Figure 2). Importantly, control students in Year 1 who went on to use Frax in Year 2 also showed significant achievement gains, fully catching up to or exceeding their Year 1 matched peers (Table 3). Conclusions: EdTech has the potential to support significant learning gains for all learners while alleviating the heavy burden associated with delivering individualized instruction. The current study provides evidence that the Frax program is an efficient and effective way to support fractions knowledge and math achievement. Efforts to control for potential bias were used in the current study, including creating equivalent groups, but future research should be conducted using an experimental methodology with students from multiple districts. Continued efforts to measure program efficacy and support evidence-based implementations must be collaboratively embraced by vendors, researchers, and school administrators.
Descriptors: Mathematics Achievement, Game Based Learning, Educational Technology, Fractions, Intervention, Program Effectiveness, Elementary School Students, Mathematics Education, Student Improvement, Academic Ability, Grade 3, Grade 4, Suburban Schools, Public Schools
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education; Early Childhood Education; Grade 3; Primary Education; Grade 4; Intermediate Grades
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A