Background: Randomised controlled trials (RCTs) have long been considered the gold standard in education research. Federal funds are allocated to evaluations that meet What Works Clearinghouse standards; RCT designs are required in order to meet these standards without reservations. Schools seek out interventions that are research based, in other words, interventions that have demonstrated effects in RCTs, creating a cycle that incentivises RCT research. However, RCTs pose tremendous obstacles for school-based participants. Methods: This paper presents a descriptive case study of an RCT of an adolescent reading intervention implemented in four districts over three years, drawing on interviews and document analysis to illustrate the challenges that participation in an RCT poses for schools. Results: In two participating districts, the random assignment requirement caused major difficulties with recruitment and retention. Of the 15 schools that began implementation in fall of year 1, 11 left the study by fall of year 2. Of those 11, eight explicitly stated that the random assignment requirement was the reason they withdrew. Stakeholders expressed that they felt it was unethical to withhold the intervention from some students. Conclusions: This rate of attrition casts doubt on the external validity of the programme evaluation, as these schools are likely to systematically differ from schools that met the requirements of random assignment. Suggestions are offered for alternative methods that may be used to evaluate promising interventions, in ways that are better aligned with real-world educational contexts.