Rigorous research provides information that will allow ECE (Early Childhood Educator) programs to select interventions that have a scientifically based track record of effectiveness in increasing teachers' skills and teaching quality. This paper shares implementation and impact results as well as the lessons learned in conducting an evaluation of an intervention developed to increase ECEs knowledge and skills and ultimately improvements in children's language and readiness skills as a result. Evaluators developed the rigorous design and external evaluation plans to assess the effectiveness of this community advocacy-service delivery organization's professional development program. The evaluation was organized as two studies: (1) an "Impact Study" designed to estimate the effects of participating in the professional development program on early education providers as well as the children in their care; and (2) an "Implementation Study" designed to provide information about the way in which the professional development interventions were implemented (i.e., fidelity of implementation or fidelity to plans/model). The evaluation plan employs a cluster-randomized trial (CRT) at the classroom level, to assess the effectiveness of the program in improving ECE practice as well as children's early reading skills. In addition, the consideration of implementation context in the interpretation of the results and what should be done when planning to implement will also be discussed. As a result evaluators hope to provide important information to those planning similar studies in such programs and other complex social settings. (Contains 10 figures.)