The purpose of this paper is (1) to discuss a set of standards that can be used to evaluate potential assessment instruments; and (2) to use these standards to evaluate the American College Testing Program's College Outcomes Measures Program (ACT-COMP) and the Educational Testing Service (ETS) Academic Profile. Using the work of S. Messick (1975, 1987, 1988) on construct validity, researchers examined the substantive, structural, and external components of use of these tests by the Tennessee Higher Education Commission (THEC) and the University of Tennessee (Knoxville). The COMP was administered to 1,828 seniors, and the Academic Profile was administered to 1,173 seniors. Also, 35 seniors agreed to take both examinations. The Academic Profile was superior in its ability to differentiate accurately among students and programs. Both tests appeared to measure a single underlying construct, and analysis suggests that this construct is academic ability, not program quality. That the THEC defines effective general education in terms of test scores is particularly troubling in light of these findings. It is contended that the THEC guidelines may: (1) unjustifiably limit the substance of general education to a narrow range of learning outcomes; (2) award funds on the basis of differences that are well within the error of measurement for the examinations; and, most importantly (3) evaluate programs on the basis of the students they attract rather than quality of the education those students receive. Eleven tables of comparisons and one figure illustrating Messick's concept of validity are provided. (SLD)