The impetus for test-based accountability systems is to improve the educational opportunities afforded to all students so as to improve their learning; therefore, integral to the validity argument of these systems is the appraisal of test-based inferences and decisions in terms of their consequences. Both positive and negative consequences of test-based decisions have different effects on different groups of students and in different schools, and these differential effects need to be examined as part of the validity argument and in addressing fairness issues. This paper addresses intended and potentially unintended consequences of test-based accountability systems in the validity argument. Legislation for test-based accountability systems, as well as studies evaluating their consequences, is discussed. A conceptual framework that provides a principled approach for evaluating both intended and unintended consequences of assessment and accountability systems, including those that arise due to using tests as policy levers for educational change, is provided. [This lecture was presented at the William H. Angoff Memorial Lecture at the National Press Club (17th, Washington, D.C., Dec 16, 2019).]