The National Assessment of Educational Progress (NAEP) Validity Studies Panel undertook a systematic analysis to consider the domain of validity threats to NAEP and to identify the most urgent research priorities. A framework of six broad categories was developed: (1) the constructs measured within each of NAEP's subject domains; (2) the manner in which these constructs are measured; (3) the representation of the population; (4) the analyses of data; (5) the reporting and use of NAEP results; and (6) the assessment of trends. Panel subcommittees prepared papers laying out the critical validity issues in each area. These papers, which are presented in chapters 2 through 7 of this report, are: (1) "Validity Issues Representing Populations" (Donald H. McLaughlin, Peter Behuniak, and James R. Chromy) (Chapter 4); (2) "Issues and Recommendations on NAEP Data Analysis" (R. Darrell Bock, Albert Beaton, and Gerald DeMauro) (Chapter 5); (3) "Validity and Utility Issues in NAEP Reporting and Data Releases" (Frances B. Stancavage and Ina V. S. Mullins) (Chapter 6); and (4) "Estimating Trends from NAEP Scores: Rationale and Research Directions" (David Grissmer, Albert E. Beaton, and Larry Hedges) (Chapter 7). The panel reviewed these papers and set priorities in each area by a consensus process. Sixteen suggested studies or areas of study were identified. Four stood out as essential, nine others were considered "highly needed," and three were found to be less important. The panel indicated unanimously that studies are essential to evaluate the validity aspects of NAEPs new role under the No Child Left Behind Act. The appendix to chapter 4 is attached. (Contains 6 figures and 26 tables.) (SLD)