Researchers explored how 13 states in which policymakers have adopted an A-F school letter grade accountability system performed on the National Assessment of Educational Progress (NAEP) post-policy implementation. Researchers found mixed results, with approximately half of these 13 states increasing achievement post-policy, and the other approximately half not, evidencing that states' adoptions of similar A-F policies may not realize increased student achievement as intended and, rather, yield more-or-less random results. Policy implications are discussed, also given seven of these 13 states are Republican/Republican-leaning and one is Democratic/Democratic-leaning, which also has implications for others looking to potentially adopt such relatively conservative, educational policy approaches towards reform.