In recent years, there has been increased attention regarding systems-level implementation to support the sustained use of evidence-based interventions and supports in authentic early childhood settings. With this comes a need to accurately measure implementation fidelity of the critical features within a framework as well as individual practices. Program-Wide Support for Pyramid Model Implementation (PWS-PMI) provides an approach for early childhood programs to develop such a framework that can underpin evidence-based practices in their classrooms. This article describes an evaluation of the technical properties of the Supporting Program-wide Implementation Fidelity Instrument (SPIFI), a fidelity tool that was developed to be used by typical evaluators to measure PWS-PMI in these settings. Findings suggest that the instrument reliably demonstrated construct validity when used by typical evaluators to assess PWS-PMI and provides initial validation of the SPIFI as an objective measure for use in evaluative research and technical assistance.