The quality of interventions for children with autism has improved thanks in part to the widespread dissemination of evidence-based practices (EBPs); however, teachers still report challenges developing focused interventions targeting the core challenges of students with autism. Tested in three randomized trials, COMPASS is a consultation-based implementation strategy that prioritizes shared decision-making in the development of goals and intervention plans using an evidence-based practice in psychology approach. To successfully train COMPASS consultants, a 16-item Intervention Plan Quality Scale (IPQS) was developed and tested with a set of nine school-based COMPASS-trained consultants who provided a total of 28 consultations. Results revealed that the IPQS had acceptable reliability and concurrent validity and was successful in helping consultant trainees develop high-quality plans over four feedback sessions. Overall, the IPQS was helpful for fidelity monitoring and appears to partially mediate child goal attainment outcomes through teacher adherence implementing the intervention plans.