ERIC Number: ED503919
Record Type: Non-Journal
Publication Date: 2008-Dec
Pages: 145
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Using Student Data to Improve Teaching and Learning: Findings from an Evaluation of the Formative Assessments of Students Thinking in Reading (FAST-R) Program in Boston Elementary Schools
Quint, Janet C.; Sepanik, Susan; Smith, Janell K.
MDRC
Formative assessments--assessments that measure what students do and do not know, so that teachers can modify their instruction accordingly--have been widely hailed as a potential vehicle for improving student achievement. Yet little solid research evidence exists about their effectiveness, especially in reform-rich school districts. This study examines the effects of the Formative Assessments of Student Thinking in Reading (FAST-R) initiative in the Boston Public Schools system (BPS), where the use of data to improve instruction is a general priority of the school district. The study looks at changes in reading scores over time at 21 BPS schools that operated FAST-R during the 2005-2006 and 2006-2007 school years and changes at a group of comparison schools serving demographically similar students during the same period. The MDRC evaluation includes process and impact analyses. The process analysis found that teachers at the FAST-R schools who took a survey administered as part of the study reported that the professional development they received from the BPE FAST-R coaches was helpful and contributed to their understanding of data and their ability to work with students. At the same time, while the intervention was implemented as intended (it was meant to be flexible and to provide as much or as little coaching to individual schools as administrators and teachers sought), it was not very intensive; the majority of survey respondents spent only one to five hours with the FAST-R data coach during the 2006-2007 school year. Moreover, comparison school teachers who took the survey reported receiving at least as much professional development as their FAST-R counterparts, were as likely to find it useful, and spent as much or more time analyzing data, including data from other (non-FAST-R) formative assessments. The impact analysis examines the effects of FAST-R on the reading test scores of third- and fourth-graders. FAST-R's impacts on student achievement--that is, the difference that FAST-R made over and above what was going on in the comparison schools--are generally positive but not statistically significant, as measured by MCAS reading scores. In other words, these differences could have arisen by chance. Effects on another measure of student reading, the Stanford Achievement Test, are more mixed but are also not statistically significant. While FAST-R schools put in place a particular model of data utilization, other BPS schools were pursuing similar goals, and this fact, along with the intervention's lack of intensity, may have undercut the likelihood that it would generate substantial and statistically significant impacts in this evaluation. Thus, this single study in a single district is not the last word on the potential of FAST-R. Much remains to be discovered about how teachers can best learn to use data to improve their instruction and boost the achievement of their students. Following an Overview, Preface, and an Executive Summary, this report is organized into four chapters. Chapter 3 discusses the professional development activities in FAST-R and non-FAST-R schools highlighted by the findings of the principal and teacher surveys. The chapter also considers how teachers perceived the utility of the FAST-R intervention for their instructional practices. Chapter 4 describes the findings from the impact analysis of FAST-R with regard to student achievement, exploring the range of student outcomes on the MCAS and the SAT-9 reading assessments. In addition, the chapter reports on an analysis to measure the impact of FAST-R on students' ability to make inferences and find evidence while reading. Lastly, subgroup analyses to compare the effect of FAST-R on various groups of students (by, for example, gender and socioeconomic status) are discussed. Chapter 5 presents the overall conclusions that may be drawn from the study's analyses and their implications for the use of formative assessments and data-driven instruction to improve reading skills. Appended are: (1) The Analytic Model Used in the FAST-R Impact Analysis; (2) List of FAST-R and Non-FAST-R Schools; (3) Subgroup Analyses of the Effects of the FAST-R Program; and (4) Sample of FAST-R Assessment Student and Teacher Materials. (Contains 28 tables, 5 figures, and 3 boxes.)
Descriptors: Formative Evaluation, Student Evaluation, Reading Tests, Data, Public Schools, Elementary Schools, Professional Development, Intervention, Reading Achievement, Teacher Surveys, Grade 3, Grade 4, Educational Change, Urban Schools, Elementary School Students, Comparative Analysis, Elementary School Teachers, Teacher Attitudes
MDRC. 16 East 34th Street 19th Floor, New York, NY 10016-4326. Tel: 212-532-3200; Fax: 212-684-0832; e-mail: publications@mdrc.org; Web site: http://www.mdrc.org
Publication Type: Reports - Evaluative
Education Level: Elementary Education; Grade 3; Grade 4
Audience: N/A
Language: English
Sponsor: William and Flora Hewlett Foundation
Authoring Institution: MDRC
Identifiers - Location: Massachusetts
Identifiers - Assessments and Surveys: Massachusetts Comprehensive Assessment System; Stanford Achievement Tests
Grant or Contract Numbers: N/A
IES Cited: ED559916; ED559928