The field of education relies heavily on instructional coaches to build teacher capacity in the implementation of evidence-based practices (EBPs). Although observation tools are commonly used to measure the fidelity of implementation by teachers, fewer tools are available to identify specific coaching behaviors used during in situ coaching sessions focused on supporting teachers with implementing EBPs. The current study aims to develop and provide preliminary evidence of the reliability and validity of a coaching observation tool. This study used an iterative process to develop the initial and revised draft of the tool through a literature review and expert input. Two pilot studies were conducted to determine the reliability of the tool. Inter-rater agreement improved between Pilot 1 and Pilot 2, with room for further improvement. We recommend additional research to further examine tool reliability and relevance for describing coaching practices within the context of other EBPs, interventions, and programs.