This study aimed to comprehensively investigate the impact of a word-processor on an ESL writing assessment, covering comparison of inter-rater reliability, the quality of written products, the writing process across different testing occasions using different writing media, and students' perception of a computer-delivered test. Writing samples of 42 international students taking two tests, a paper-and-pencil based ESL placement writing test and a computer-based one, were analyzed. The results showed that while there was no significant mean difference in the holistic ratings across test occasions, all the analytic components of the computer-generated essays were marked significantly higher than those of the paper-based essays. In a holistic measurement, rater reliability was significantly higher in word-processed essays than in paper-written essays. A student survey revealed that habitual computer writers preferred a computer-delivered writing mode to a traditional testing mode. Suggestions and implications for further study for implementing computer-based ESL writing tests are discussed. (Contains 7 tables.)