With the expansion of performance-based incentives, there is growing interest in understanding how they affect leadership quality and composition. This paper studies a reform in a large urban public school district that ties principal compensation to their performance. Using a regression discontinuity design, I investigate the effect of categorical evaluation ratings on principal composition and future school leadership effectiveness for equally productive principals. Results indicate that principals with rating scores that place them just below the higher rating cutoff are more likely to turnover, exit the district, and are less likely to be promoted. However, I do not find much evidence of changes in future leadership productivity in the schools where principals fall below the cutoff. Principal transitions around the rating boundary may be driven by the "label effect" (changes in principal motivation, district decisions, and signal sent to other employers) and/or the "salary effect". Exploiting the information that crossing the higher rating threshold does not result in a salary change for all principals, I document that the transition effects are concentrated within the group that does experience a salary change at the rating boundary. This highlights the significance of the role salary plays in transition decisions. Findings from this paper have important policy implications regarding the implementation of performance-based incentive programs.