Purpose: The aim of this study was to advance the analysis of written language transcripts by validating an automated scoring procedure using an automated open-access tool for calculating morphological complexity (MC) from written transcripts. Method: The MC of words in 146 written responses of students in fifth grade was assessed using two procedures: (1) hand-coding of words containing derivational morphemes by trained scorers and (2) an automated analysis of MC using Morpholex, a newly developed web-based tool. Correlational analysis between the different MC calculations was examined to consider the relation between hand-coded derivational morpheme counts and the automated measures. Additionally, all MC measures were compared to a previously gathered rating of writing quality to consider predictive validity between the automated Morpholex score and teachers' ratings of writing quality. Results: Automated measures of MC had a strong relation (r = 0.63) with hand-coding of the number of words with derivational morphemes. Additionally, the number of derivational and inflectional and derivational morphemes accounted for a significant amount of the variation in teachers' overall ratings of writing quality. Conclusion: Automated scoring of MC has potential utility as a valid alternative to hand-coding language samples, which may be valuable for progress monitoring of growth in complexity across repeated samples and measuring components that influence perceived quality of academic writing. [This is the online version of an article published in "American Journal of Speech Language Pathology."]