Direct Behavior Rating (DBR) has emerged as a useful assessment method to identify behavioral risk and monitor behavioral progress. The development of an online training module has provided an option for low cost training accessible to a wide variety of users at their convenience. Although evaluation of the DBR online training module has demonstrated improved rater accuracy following completion, reliability of obtained scores has not been fully explored. An improvement in reliability of scores is desired to allow for flexible use across raters and increased efficiency in decision-making. In this study, four teachers simultaneously rated the activity engagement and disruptive behavior of six children during their ice skating activity at a summer day camp. Ratings were analyzed within a generalizability theory (GT) framework. Results suggest that completion of the online training module may result in some improvement in the reliability of data generated from DBR-SIS. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]