This study examines whether "My Math Academy," a digital game-based learning environment that provides personalized content and adaptive embedded assessments, can improve math knowledge of Transitional Kindergarten and Kindergarten students. A cluster randomized study was conducted with 20 classrooms, 10 of which were randomly assigned to use "My Math Academy" for 12-14 weeks. Results of hierarchical linear models indicated that after using the app for about 5 hours, treatment students significantly outperformed the control group in math knowledge and skills as measured by TEMA-3. "My Math Academy" produced the greatest learning gains in students who began with a moderate level of math knowledge and on the most difficult skills assessed. Additionally, the more games students played, the greater the learning gains they experienced. Teacher surveys revealed that "My Math Academy" produced positive impacts on students' interest and self-confidence in learning math. Teachers also recognized "My Math Academy" as a user-friendly, valuable learning resource that they will continue to use. The study results will inform ongoing development of "My Math Academy," including tools to enable teachers to help children with low levels of prior math knowledge.