The purpose of this study was to develop, test, and use an instrument to measure the effectiveness of a community-based, mandated drug education intervention program, Youthful Offenders Program (YOP), targeting college students at risk. A total of N = 350 students voluntarily agreed to participate in an evaluation of program effectiveness using the Alcohol Use Disorders Identification Test (AUDIT) and a newly developed measure to compare changes in substance related behaviors over time. Results revealed sound psychometric structure for the new scale for use in program evaluation; results suggest that YOP significantly enhanced safety practices and reduced hazardous drinking and recidivism, but not avoidance of risky behaviors.