School accountability systems have been roundly criticized for their narrowness, being largely informed by standardized achievement scores. With the passage of the Every Student Succeeds Act, accountability systems may become more multifaceted in coming years. This study examines pilot results of an alternative school quality framework--one generated with stakeholder input and incorporating a wide range of measures. Comparing this alternative system against the current state accountability system, we find that the two systems produce somewhat similar overall rankings. However, we find substantial variation across particular measures within the alternate framework, indicating that even if the current state system adequately captures overall school quality, it obscures important differences--strengths and weaknesses--across schools. Policy implications of these findings are explored