Principals can affect several consequential schooling processes and outcomes. However, their effectiveness varies substantially and is distributed across schools inequitably, underscoring the importance of effective principal professional development (PPD), which begins by using needs assessments to inform PPD content. A researcher-practitioner partnership assessed principal needs via monthly teacher surveys about specific teacher evaluation skills. While internal consistencies were high, test-rest reliabilities were low, implying that reports regarding the quality of specific principal practices may fluctuate substantially over short periods, potentially hampering the design of effective PPD.