The use of reliable and valid tools to evaluate theory-based health interventions is widely accepted as critical to the behavioral sciences. However, quite often the psychometric properties of survey instruments used to evaluate behavioral interventions are not published, limiting our understanding of how the theory works. The purpose of this study was to present the results of psychometric analyses on measures used to evaluate an HIV/AIDS prevention program for youth in Tanzania. We first began by pilot-testing a self-administered questionnaire with sixth and seventh-grade students in nine schools in 11 villages throughout the Kahe ward of Tanzania (n = 524). Theory-based scales tapped into intrapersonal, social and attitudinal factors. Reliability was evaluated by examining internal consistency, while validity was determined by examining content, construct and discriminant validities. The results indicate that the scales constructed are reasonably reliable and valid measures. Use of these measures should be considered when designing, implementing, and analyzing interventions targeting youth in sub-Saharan Africa.