The purpose of this study was to evaluate the internal consistency reliability, validity, and diagnostic accuracy of Social, Academic, and "Emotional Behavior Risk Screener--Teacher Rating Scale" (SAEBRS) scores. Teachers (n = 68) universally screened 1,242 elementary students using two measures: the SAEBRS and the "Behavioral and Emotional Screening System" (BESS). Multilevel analyses indicated that although SAEBRS scores were internally consistent at the overall level, reliability suffered for certain SAEBRS scores at the between-group (classroom) level. Multilevel correlational analyses revealed moderate-to-large and statistically significant relations between SAEBRS and BESS scores at the overall, between-group, and within-group levels. Follow-up Fisher's z tests revealed a pattern of convergent and discriminant relations in accordance with theory-driven expectations. Receiver operating characteristic (ROC) curve analyses supported the diagnostic accuracy of each SAEBRS scale. Further examination of findings relative to prior research suggested SAEBRS diagnostic accuracy is moderated by the type of criterion measure under consideration.