Cluster Randomized Trial (CRT) designs are inherently multilevel and reflect the hierarchical structure of schools and the wider education system. To capture this multilevel nature, CRTs are commonly analysed using multilevel (or hierarchical) linear models. It is fairly common for CRT designs and analyses to include school and individual/pupil levels but the inclusion of a within-school level (e.g. class or teacher) is rare. This paper presents an examination of the methodological and empirical implications of not including a class level within CRT designs and impact analyses, and is organised into five sections: (1) introduction of the research questions; (2) presentation of a summary of three level CRTs that included school, classroom and pupil levels; (3) focus on EEF maths trials given that this subject has the greatest incidence of pupil segregation/setting; (4) examination of the methodological theory of RCT and CRT designs; and (5) reflection on the empirical data from the second and third sections from a methodological perspective before reflecting more widely about whether classroom level 'matters' in the design of educational CRTs.