The authors examined the ecological validity of using observation-based scores to evaluate individual teachers' effectiveness, mirroring their use by school administrators. Using the Classroom Assessment Scoring System, the authors asked (a) how similar are teachers' emotional support, classroom organization, and instructional support scores from each other; and (b) to what extent are these scores stable from day-to-day? Using a time-series, single-case methodology, the authors examined patterns of eight Kindergarten teachers' performance in all lessons within a 4-5 week, thematically coherent science unit. Thus, the opportunity for identifying within-teacher stability was maximized. Teachers' instructional quality was significantly less effective than their emotional supportiveness and classroom organization. Furthermore, teachers' effectiveness ratings varied markedly from one lesson to the next, for all three dimensions.