This paper considers one challenge from recent emphasis on rigorous research for decision-making in education: the interpretation of non-replicated findings. Under Clinton and Bush, the federal government called for and funded highly rigorous research. The "gold-standard" for research was randomized-control trials (RCTs), which supposedly offered clear answers about program efficacy. Such clear answers were meant to be used by stakeholders in making decisions about adopting programs. During 2005-2008, the authors conducted an RCT meant to measure the impact of a reading program in the afterschool setting, with the goal of comparing the effectiveness of that program with regular school offerings. The study covered three school years, with slight variations in both the delivery of the intervention and the design of the research, resulting in three essentially discrete studies conducted within a single school district. The authors hoped by replicating the study in one context, they would be able to draw conclusions about the effectiveness of the reading program. This paper, therefore, addresses the following questions: (1) How may contradictory findings be used in program design and implementation? (2) How may the conversation among practitioners, policymakers and researchers be expanded to acknowledge both consistency and contradictions in robust research studies? and (3) How may we better communicate what contradictory findings suggest about practice and program development to practitioners?