This mixed-method study examined the impact of an online formative assessment tool on the language and early literacy skills development of 204 3-4-year-old children enrolled in Head Start. Twenty-four classroom and children participated in a two-group randomized control trial, in which classrooms in the treatment condition implemented weekly assessment of oral language, alphabet recognition, and phonological awareness using the Curriculum-Based Decision Making tool, and based upon assessment output, made minor modifications or instituted simple interventions for individual children. Children in the "business as usual classrooms" followed their lesson plans as traditionally done in their program. Based upon pre and post-assessment of early literacy skills, children in the treatment condition significantly outperformed their peers on both measures. Furthermore, teachers learned to use and embraced performance data to rapidly make instructional decisions and follow-up on children's individual and group progress.