ERIC Number: EJ843858
Record Type: Journal
Publication Date: 2006-Nov
Pages: 50
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1540-2525
EISSN: N/A
Does It Matter if I Take My Writing Test on Computer? An Empirical Study of Mode Effects in NAEP
Horkay, Nancy; Bennett, Randy Elliott; Allen, Nancy; Kaplan, Bruce; Yan, Fred
Journal of Technology, Learning, and Assessment, v5 n2 Nov 2006
This study investigated the comparability of scores for paper and computer versions of a writing test administered to eighth grade students. Two essay prompts were given on paper to a nationally representative sample as part of the 2002 main NAEP writing assessment. The same two essay prompts were subsequently administered on computer to a second sample also selected to be nationally representative. Analyses looked at overall differences in performance between the delivery modes, interactions of delivery mode with group membership, differences in performance between those taking the computer test on different types of equipment (i.e., school machines vs. NAEP-supplied laptops), and whether computer familiarity was associated with online writing test performance. Results generally showed no significant mean score differences between paper and computer delivery. However, computer familiarity significantly predicted online writing test performance after controlling for paper writing skill. These results suggest that, for any given individual, a computer-based writing assessment may produce different results than a paper one, depending upon that individual's level of computer familiarity. Further, for purposes of estimating population performance, as long as substantial numbers of students write better on computer than on paper (or better on paper than on computer), conducting a writing assessment in either mode alone may underestimate the performance that would have resulted if students had been tested using the mode in which they wrote best. (Contains 13 endnotes, 14 tables, and 2 figures.)
Descriptors: Writing Evaluation, Writing Tests, Computer Assisted Testing, Program Effectiveness, Grade 8, Writing Skills, Correlation, Comparative Analysis, Educational Assessment, Elementary School Students, Test Items, Educational Testing, Student Evaluation, Internet, Educational Technology, Test Format, Test Content, Comparative Testing, Printed Materials, Methods Research, Test Bias, Laptop Computers, Computer Literacy
Technology and Assessment Study Collaborative. 332 Campion Hall, Boston College, Chestnut Hill, MA 02467. Tel: 617-552-4521; Fax: 617-552-8419; e-mail: jtal_editor@bc.edu; Web site: http://escholarship.bc.edu/jtla
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; Elementary Secondary Education; Grade 8
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A