"LifeSkills Training" (LST) is a popular drug prevention program shown to be effective at preventing youth substance use. The Middle School Health Survey (MSHS) is a questionnaire that was created by the developers of the LST curriculum to measure changes in knowledge and attitudes about drugs and life skills among youth. Despite the widespread use of the MSHS, the measurement quality of the instrument is largely unknown because no prior research has evaluated its psychometric properties. This study evaluated the reliability of scores and factor structure of the MSHS, which was administered to 986 students (Grades 6-8) before the implementation of the LST curriculum. Results indicate that one of the subscales had poor internal consistency reliability as measured by Cronbach's alpha, which is likely due to restricted variability in item responses. Exploratory and confirmatory factor analytic procedures indicated that a three- or four-factor solution fit the data; however, fewer than half of the items had loadings above the acceptable threshold, indicating that the MSHS requires modification to meet acceptable standards of measurement quality. This paper fills a void in the literature and informs if and which modifications to the MSHS might be necessary.