Direct behavior ratings (DBRs) combine aspects of both systematic direct observation and behavior rating scales to create a feasible method for social behavior assessment within a problem-solving model. The purpose of the current study was to examine whether accuracy of DBRs was affected depending on the behaviors selected to be rated using a DBR. Specifically, the impact target behavior wording (positive vs. negative) and degree of specificity by which the behaviors were defined were investigated. Participants (N = 145), assigned to 1 of 4 experimental conditions, were asked to view video clips of a classroom setting and rate target student behavior. Results indicated that the wording and specificity of behavior included on a DBR may influence the accuracy of ratings. The most accurate ratings of academically engaged behavior occurred when the behavior was defined with a global definition and positive wording. In contrast, the most accurate ratings of disruptive behavior were obtained with a global definition and either positive or negative wording. Limitations, implications, and future directions are discussed. (Contains 2 tables and 2 figures.)