NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED663005
Record Type: Non-Journal
Publication Date: 2024-Sep-19
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Are Bundled Accommodations More Beneficial than Individual Accommodations? Effectiveness of Breaks and Extended Time Accommodations
Xin Wei
Society for Research on Educational Effectiveness
This investigation scrutinizes the relative effectiveness of individual versus bundled accommodations -- specifically, breaks and extended time (ET) -- on the performance, behaviors, and attitudes of eighth-grade students with disabilities during the 2017 National Assessment of Educational Progress (NAEP) digital math test. Through a detailed propensity score analysis, findings reveal that while integrating breaks with ET diminishes performance compared to the application of ET alone, the inclusion of ET with breaks substantially enhances outcomes compared to utilizing breaks in isolation. Sole reliance on breaks correlates with inferior performance metrics, reduced item completion rates, fewer test-taking actions, heightened perceptions of time pressure, and decreased interest in mathematics. These results underscore the necessity to reassess current accommodation policies to better support students with disabilities, advocating for a more nuanced approach to the provision of test accommodations. Background/Context: In the evolving landscape of educational assessment, ensuring assessments accurately reflect a student's knowledge and skills rather than the limitations imposed by disabilities remains a formidable challenge, especially within the context of digital assessments. This challenge is magnified by federal mandates, such as the Individuals with Disabilities Education Act (IDEA, 2004), which necessitate the inclusion of students with disabilities (SWDs) in large-scale assessments and allow for accommodations to mitigate the impact of disabilities (Sireci, Scarpati, and Li, 2005). Among various accommodations, extended time (ET) and breaks are two of the most commonly used accommodations designed to offset disabilities' impact, aiming for equitable assessment conditions (Wei & Zhang, 2023; Abedi et al., 2008; Lai & Berkeley, 2012). These accommodations are essential in enhancing the reliability, validity, and fairness of assessment outcomes by eliminating construct-irrelevant barriers, thereby enabling SWDs to demonstrate their competencies equitably (AERA, APA, & NCME, 2014). Purpose/Objective/Research Question: This study aims to evaluate the individual and combined effects of breaks and ET accommodations on the mathematical test performance, test-taking behavior, and attitudes towards mathematics of SWDs. It seeks to ascertain whether bundled accommodations, specifically the combination of breaks and ET, provide superior or inferior outcomes compared to each accommodation applied independently. Setting: The research utilized data from the 2017 National Assessment of Educational Progress (NAEP) digital math test, representing a diverse cross-section of eighth-grade students across the United States. Population/Participants/Subjects: The study sample comprised approximately 2,750 eighth-grade students with disabilities who participated in the 2017 NAEP math assessment. These students received accommodations in varying combinations, including breaks, ET, or both, as determined by their Individualized Education Programs (IEPs). Intervention/Program/Practice: The intervention examined in this study involves the provision of two common test accommodations (breaks and ET) to SWDs during their participation in a standardized math assessment. The study compares the efficacy of these accommodations both individually and in combination. Research Design: Employing a quasi-experimental design through propensity score analysis (Rosenbaum & Rubin, 1983), this study aimed to reduce selection bias and approximate the conditions of randomized controlled trials. This methodological approach enhances the credibility of the findings by ensuring that the comparison groups are well-matched on several covariates. Data Collection and Analysis: Data collection involved the use of NAEP's restricted-use dataset, which includes detailed response, process, and survey data from the 2017 eighth-grade math test. The analysis focused on evaluating the impact of breaks, ET, and their combination on a range of outcomes including test performance, test-taking behaviors (e.g., time spent on items, number of items completed), and attitudes towards math. Findings/Results: The analysis revealed that students receiving both breaks and ET performed significantly worse than those receiving ET alone, suggesting that the combined accommodations may introduce unintended negative effects on performance. This paradox spotlights the complex art of balancing effective support against the risk of inadvertently hampering academic performance. While breaks aim to mitigate stress and fatigue, they might also, perhaps unexpectedly, fragment a student's concentration and workflow -- a detrimental side effect in subjects demanding consistent engagement like mathematics. Intriguingly, the bundled group, in contrast to those with only breaks, exhibited heightened engagement and interaction with the assessment. This is a beacon of the psychological and behavioral advantages tied to ET, supporting Wei and Zhang's (2023) findings. The bundled group's increased time on tasks, frequent use of assistive tools, and thoroughness in attempting questions suggest that while this combination might not directly amplify academic performance, it fosters a more immersive and involved testing experience. The 'breaks only' strategy in timed assessments, however, emerges as a double-edged sword. Our data indicates that breaks alone may not suffice in supporting SWDs, potentially even detracting from their performance and overall test-taking experience. The risk lies in the unaccounted time during breaks, which may culminate in heightened stress and a sense of urgency -- outcomes diametrically opposed to the intended purpose of breaks. This insight crystallizes the notion that while breaks are indeed a valuable resource, their integration with other accommodations like ET is crucial for optimizing effectiveness in both assessment and instructional settings. Conclusions: The study's findings challenge the efficacy of bundling breaks with ET as an accommodation strategy for SWDs in digital math assessments. While ET alone appears beneficial, its combination with breaks does not confer additional advantages and may, in fact, hinder performance. These insights call for a reevaluation of accommodation policies, emphasizing the need for accommodations to be tailored to the unique needs of individual students to optimize their assessment experiences and outcomes.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education; Grade 8; Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Laws, Policies, & Programs: Individuals with Disabilities Education Act
Identifiers - Assessments and Surveys: National Assessment of Educational Progress
Grant or Contract Numbers: N/A