Test-based educational accountability systems have considerable appeal to politicians, policymakers, and the general public. Such systems have been widely used by states for more than a decade and with the enactment of the No Child Left Behind Act of 2001 all states must now implement an accountability system that uses results from assessments in mathematics and English/language arts and is administered each year in grades 3 through 8 plus one high school grade. A wide variety of test-based accountability systems of states and as required by NCLB are described and their strengths and weaknesses are evaluated. It is argued that the sanctions for schools that are part of the accountability system require causal inferences about school effectiveness. It is concluded, however, that basing causal inferences about school quality on the results that can be obtained from the existing school accountability systems is not scientifically defensible. It would be better to view accountability results as a source of descriptive information about schools and the basis of hypotheses that can be evaluated by gathering addition information about instructional staff and practice. (Contains 2 figures and 1 table.)