This paper reports the results of an analysis of the relationship between scorer behaviors and score variability. Thirty-six essay scorers were interviewed and asked to perform a think-aloud task as they scored 24 essays. Each comment made by a scorer was coded according to its content focus (i.e. appearance, assignment, mechanics, communication,…

This study demonstrates the advantages of using a constrained optimization algorithm to explore the optimal number of prompts, modes of discourse, and raters for achieving an acceptable level of reliability during a direct writing assessment. Writing samples elicited from 50 college students were rated by 3 graduate students and the scores…

This supplement is designed to be used in conjunction with the elementary level educator's guide to the Texas Essential Knowledge and Skills (TEKS) curriculum. The "Educator's Guide" shows the components of the TEKS that are eligible for testing, and this supplement provides information about the types of passages, test items, and…

The goal of this series of guides is to show Texas educators what components of the Texas Essential Knowledge and Skills (TEKS) curriculum are eligible for testing on the Texas Assessment of Academic Skills (TAAS) and end-of-course tests. This elementary level guide contains information for subjects tested in grades 3 through 6. The sections are:…

The goal of this series of guides is to show Texas educators what components of the Texas Essential Knowledge and Skills (TEKS) curriculum are eligible for testing on the Texas Assessment of Academic Skills (TAAS) and end-of-course tests. This middle school guide contains information for subjects tested in grades 6 through 8. The sections are: (1)…

Effects of using different combinations of internal and external matching variables were examined using a generalized Mantel-Haenszel statistic, a technique based on meta-analysis, and logistic discriminant function analysis with data from a writing assessment for over 4,200 8th graders. Results did not support use of an external matching…

Rater errors (rater severity, halo effect, central tendency, and restriction of range) are described, and criteria are presented for evaluating rating quality based on a many-faceted Rasch (FACETS) model. Ratings of 264 compositions from the Eighth Grade Writing Test in Georgia by 15 raters illustrate the discussion. (SLD)

Details the history of California's experience with large-scale writing assessment. Describes the introduction, development, and scoring of the California Assessment Program, later renamed the California Learning Assessment System (CLAS). Discusses controversies over removal of certain writing prompts and objections to CLAS. Notes that funding for…

This article decries the negative effects of the standards movement and nationwide high-stakes testing on the teaching of creative writing as teachers teach to the test and formulaic writing is preferred over truly generative writing. The relationship of good writing to good thinking is stressed. (Contains references.) (DB)

Automated Essay Scoring (AES) is defined as the computer technology that evaluates and scores the written prose (Shermis & Barrera, 2002; Shermis & Burstein, 2003; Shermis, Raymat, & Barrera, 2003). AES systems are mainly used to overcome time, cost, reliability, and generalizability issues in writing assessment (Bereiter, 2003; Burstein,…

The Work Keys (trademark) system of the American College Testing program has combined assessments of listening and writing into a single test administration. The Listening and Writing assessment uses independent scorings of the examinee's written responses to audiotaped prompts to assess both listening and writing skills separately. This paper…

This booklet illustrates and compares three important standards (levels of expectation) set for diploma examination writing in Social Studies 30: excellent, proficient, and satisfactory. Sample papers are used to illustrate the scoring criteria for group leaders and markers are presented, along with accompanying commentaries, or rationales, that…

This digest offers guidelines to help teachers prepare students with disabilities to succeed on state and district writing assessments. Teachers are urged to use the three principles of effective writing instruction: (1) use a basic framework of planning, writing, and revision; (2) instruct students in steps of the writing process and the features…

Specialists in the field of large-scale, high-stakes writing assessment have, over the last forty years alternately discussed the issue of maximizing either reliability or validity in test design. Factors complicating the debate--such as Messick's (1989) expanded definition of validity, and the ethical implications of testing--are explored. An…

The goal of this series of guides is to show Texas educators what components of the Texas Essential Knowledge and Skills (TEKS) curriculum are eligible for testing on the Texas Assessment of Academic Skills (TAAS) and end-of-course tests. This booklet contains information for tests administered beginning in the 10th grade as graduation…