This paper describes the intentions of educators and policymakers who implemented a new assessment strategy in Kentucky in 1991, along with its associated shortcomings and one notable success. The principle behind the assessment system was that rather than having an on-demand assessment that would inefficiently replicate the information that was already known, Kentucky would have an assessment that simply collected the already-known information. The premise was that teachers were supposed to be running their classrooms in a way that naturally resulted in assessment information, so the need for an external assessment would be changed and reduced if teachers were doing their jobs correctly. In practice, it did not work. Teachers did not change their practices much, and they did not have the needed information. Nor was there much real improvement in student achievement. The system evolved into one in which the statewide assessment program began to tell teachers what to do and how to do it. Only one part of the assessment really worked as anticipated: the writing portfolios. One reason this assessment came to work so well is that feedback from an early set of portfolios that seemed to be "way out of line" was extensively discussed and analyzed. Teacher training was revised, and teachers reduced scoring errors and elicited better work from students. Several reasons why this improvement in assessment and achievement was not possible in other subject areas are discussed. The experience of Kentucky does show that, where there is the right convergence of training, resources, and will, it is possible to create significant gains in learning. (SLD)