This paper introduces a rationale and method for constructing behavioral analytic measures of training program effectiveness that can be adopted by directors of training in diverse settings. The model includes derivation of problematic, on-the-job situations and effective responses from a target sample of trainees and builds an evaluation measure based on this derivation. The target population consists of entry level practicum students in their second year of doctoral training. The model contains five steps. In the first step, "situational analysis," a survey of the relevant characteristics of the environment is conducted with a sample of subjects currently performing in that setting. These subjects generate a detailed list of problematic situations that they have encountered while performing on the job. The next step, "response enumeration," is a sampling of the target population's common responses to these situations. The following phase, "response evaluation," uses a panel of experts to evaluate the effectiveness of the various responses to the problematic situations generated in the preceding phase. These first three steps in the process are the "criterion analysis." The next step is to construct a format (such as the Practicum Evaluation Measure) for presenting the selected situations (plus possible responses) to successive samples. The final step is to evaluate the measure using standard psychometric procedures. Details of ongoing research with this model at a psychology department training site are presented. Several program evaluation designs are recommended, to assess the effectiveness of a program for trainees with different characteristics, different supervisory formats, or different training components. (LLL)