The design and evaluation of a rating scale for program evaluation in problem-based medical curricula are described. The design process was guided by a theory of how students learn in problem-based learning as modeled by W. H. Gijselaers and H. G. Schmidt (1988). The rating scale was used in an evaluation of a new problem-based, community-oriented curriculum at the University of Sherbrooke in Quebec (Canada). A 46-item rating scale was constructed using problem-based learning theory. After each unit of the curriculum, the rating scale was administered to the 95 students in the first year of the new curriculum. Construct validity was tested by factor analysis. The internal consistency of the scale and interrater reliability were assessed. Criterion validity was investigated through a regression analysis. The 10 subscales of the rating scale appeared to cover the most relevant constructs of the problem-based approach. Subscales generally demonstrated sufficient internal consistency and interrater agreement, but criterion validity was rather low, which is attributed to the way student achievement was measured. This theory-guided approach to program evaluation contrasts sharply with procedures emphasizing "bottom-up" construction of a rating scale. Five tables present study data. (SLD)