This study illustrates a method for evaluating the accuracy of performance metrics and identifies potential institutional and student attributes that may increase the likelihood of classification errors. Longitudinal data were obtained from the Integrated Postsecondary Eduation Data System for public 4-year institutions (N = 558) to evaluate various performance metrics based on graduation rates, total credentials conferred, and total credentials per 100 full-time equivalent students. A value-added measure defined as the difference between an institution's actual and predicted graduation rate served as the reference standard. The results revealed that 18% to 56% of institutions were misclassified as effective or ineffective when using unadjusted performance metrics. Mean comparisons indicated that institutions misclassified as ineffective frequently had input disadvantages (e.g., low admissions selectivity). Some performance metrics may thus yield unacceptably high error rates in classifying institutions as effective or ineffective, which may hinder efforts to identify best practices and reward institutions that promote state and federal attainment goals.