We investigated the promise of a novel approach to formative writing assessment at scale that involved an automated writing evaluation (AWE) system called MI Write. Specifically, we investigated elementary teachers' perceptions and implementation of MI Write and changes in students' writing performance in three genres from Fall to Spring associated with this implementation. Teachers in Grades 3-5 (n = 14) reported that MI Write was usable and acceptable, useful, and desirable; however, teachers tended to implement MI Write in a limited manner. Multilevel repeated measures analyses indicated that students in Grades 3-5 (n = 570) tended not to increase their performance from Fall to Spring except for third graders in all genres and fourth graders' narrative writing. Findings illustrate the importance of educators utilising scalable formative assessments to evaluate and adjust core instruction.