Despite the popularity of automated writing evaluation (AWE) that has provoked an increased scholarly interest, synthesized research to comprehensively understand its pedagogical effects is still in paucity. To fill the gap, this study aims to meta-analyse the overall effect of AWE on learners' writing skill development and whether the effect differs from how it has been used. From activity theory (AT) perspective, this study reported results based on a meta-analysis of 33 valid effect sizes involving 2995 participants from 25 experimental and quasi-experimental studies published during 2000-22. The results of the study showed that the overall effect was g = 0.429, 95% CI [0.266, 0.592], suggesting that AWE for writing skill development is more effective than traditional methods, such as traditional classroom interaction, teacher-led interaction and peer interaction. Regarding AT-related moderators for the overall effect, different moderator effects of educational levels, proficiency levels, sample size types, target language types, intervention durations, text genre types, human-computer interaction types, feedback types and program types were reported. Informed by the results obtained, this study also provided some insights into the pedagogical implications for practice.