The Kansas State Department of Education (KSDE) accredits all public schools in the state through the Kansas Education System of Accreditation (KESA). KESA is designed to evaluate at a system level, meaning school districts, a system of schools, or an independent school not affiliated with a school district. District leaders have been frustrated by the ambiguous nature of the indicators used to evaluate system performance. This ambiguity, lack of transparency, and inconsistency produced unnecessary friction and decreased practitioners' confidence in the accreditation process--what I call the external validity threat. Many within KSDE have likewise lost confidence in its value as a tool for accountability and system improvement--what I call the internal validity threat. This capstone draws on research on the positive and negative impacts of federal accountability to glean lessons and principles for a more objective and transparent approach to accreditation. It describes and analyzes my efforts to lead a working group developing indicators to define and operationalize evidence of improvement in student performance that are more credible and transparent for all stakeholders. I explain how I first developed a theory of action focused on reducing internal and external validity threats with the goal of better positioning accreditation as an effective accountability and system improvement tool. While I initially believed that my chosen inputs and outputs were the necessary connectors to fulfill this goal, I concluded that setting up a separate accountability system with touchpoints with accreditation would be more effective. While accountability can guide system improvement, it cannot effectively drive it. System improvement ultimately depends upon the support and services provided. Accountability systems are necessary for system improvement but never sufficient. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]