Recently, the Test of English as a Foreign Language (TOEFL) changed by including a writing section that gives the examinee an option between computer and handwritten formats to compose their responses. Unfortunately, this may introduce several potential sources of error that might reduce the reliability and validity of the scores. The seriousness of these sources of error was studied by examining the quality of the ratings and the distribution of construct-irrelevant variance. Participants were 152,951 TOEFL examinees who participated in regular TOEFL administrations. Results indicate that raters have slightly better agreement for word-processed essays than handwritten essays. The generalizability analyses suggest that this is caused by the interaction of examinees x items x readers confounded with undifferentiated error. (Author/SLD)