This study used longitudinal data and a pre-post test design to examine the effect of the outcome measure of L2 writing on the accuracy rates of thirty-two international graduate students enrolled in a semester-long content-based EAP course. As part of the coursework, the students completed weekly written assignments on which they received weekly written correcting feedback (CF). Their written work on three academic writing tasks (in-class essays, in-class summaries and at-home summaries) performed twelve weeks apart (during the first and last weeks of the class) was analyzed in terms of its overall accuracy (total number of errors per 300 words), as well as grammatical and lexical errors separately. The results of this study indicate that the three outcome measures of writing performance analyzed in this study produce different estimates of L2 writers' accuracy. This study also found differences in the longitudinal changes in accuracy of L2 writing depending on the types of errors that are included in the operational definition of accuracy. These findings provide evidence for opening a discussion about the nature and technical characteristics of the outcome measures to be used as indicators of a learner's ability to produce accurate L2 texts in real-life contexts. (Contains 1 figure and 2 tables.)