Classroom observation is an accountability practice which promotes the evaluation of teachers' capacity to meet standards, improve teaching practices, and enhance student learning outcomes. Prior research has revealed that these practices are not without bias: the reliability of observation can be challenged because of classroom or observer characteristics. Unlike general education learning environments, much less is known about the reliability of observing classrooms that service students with disabilities. In this study, we evaluated the reliability of special-education classroom observation by school personnel. Using a systematic scorer design modeled after the MET Project, we examined different combinations of observers (special educator, school leader, paraprofessional, and researcher) and lessons to test the impact of prior exposure to a teacher on scoring the RELATE Tool for Special Education Classroom Observation (RELATE). Analyses of RELATE scorer reliability, comparison of school personnel and researchers on their scores' aggregates and variance, and the implications for generalization and special education classroom observation are discussed.