ERIC Number: ED595219
Record Type: Non-Journal
Publication Date: 2019-Jun
Pages: 40
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-
EISSN: N/A
Impact Evaluation of "ACCESS": A Culture Creating Effective Systems for Success
Askew, Karyl; Stevenson, Olivia; Byrd-Porter, Ginny; Auman, Wade; Lancaster, Beth; LeGrand, Takeda
Grantee Submission
"ACCESS", A Culture Creating Effective Systems for Success, was a three-year Investing in Innovation (i3) development grant funded by the Office of Innovation and Improvement, U.S. Department of Education. The "ACCESS" grant led the way for a district-wide redesign of educational practices based on the novel approach of embedding a culture of technology-based education (tech culture) within rural schools and the broader community, with an intentional effort to record, evaluate, refine, and disseminate an effective model for implementation across the state and nation. The impact evaluation included two studies that examined the effect of "ACCESS" on the academic achievement of students in grades 5-10 as measured by NC standardized End-of-Grade assessments and the American College Testing (ACT) standardized test. The middle grades study (Study 1) used a comparative short interrupted time series (CSITS) design to assess the impact of the intervention on mathematics and reading achievement in grades 5 through 8 after two years of program exposure. The high school study (Study 2) used a quasi-experimental design (QED) to assess the impact of ACCESS on ACT composite scores taken in grade 10 after two years of program exposure. For both studies, we compared the outcomes of ACCESS participants with matched samples of non-participants in other NC school districts. Propensity score matching (PSM) was used to match ACCESS samples with comparison students at baseline; baseline equivalence was established all pre-test assessment measures. Both studies met What Works Clearinghouse (WWC) Group Design Standards with Reservations. The results indicated no statistically significant differences in achievement between the "ACCESS" treatment group and the business-as-usual comparison groups as measured by the ACT composite scores taken in grade 10 or NC End-of-Grades 5-8 standardized reading and mathematics exams. To contextualize the findings, researchers considered the duration of students' exposure to "ACCESS", alignment between technology integration and NC standardized assessments, and contextual factors that may have narrowed the divide between MCS and other school districts from which comparison schools/samples were drawn. The report concludes with suggestions for future research and implications for education policy. [Funding for this report came from the U.S. Department of Education under its Investing in Innovation (i3) initiative within the Office of Innovation and Improvement through Grant U411C140128 to Montgomery County Schools.]
Descriptors: Program Evaluation, Grants, Educational Innovation, Educational Improvement, Technology Integration, Curriculum Design, Middle School Mathematics, Academic Achievement, Mathematics Achievement, Reading Achievement, Rural Schools, Program Implementation, Middle School Students, College Entrance Examinations, Standardized Tests, High School Students, Outcomes of Education, Scores, Mathematics Tests, Reading Tests, Program Descriptions, Distance Education, Fidelity, Intervention, Educational Practices, Instructional Design, Sustainability
Publication Type: Reports - Evaluative
Education Level: Junior High Schools; Middle Schools; Secondary Education; Higher Education; Postsecondary Education; High Schools
Audience: N/A
Language: English
Sponsor: Department of Education (ED), Office of Innovation and Improvement
Authoring Institution: The Evaluation Group (TEG); Montgomery County Public Schools (MCPS)
Identifiers - Location: North Carolina
Identifiers - Assessments and Surveys: ACT Assessment
Grant or Contract Numbers: U411C140128
What Works Clearinghouse Reviewed: Does Not Meet Evidence Standards
WWC Study Page: https://ies.ed.gov/ncee/wwc/Study/89892