A study evaluated the overall evaluative validity of the Four Skills Exam, a Spanish language proficiency test designed to ensure that bilingual education teachers in New Mexico can meet Spanish language demands in the bilingual education classroom. The test's construct validity was limited for several reasons. In designing a test capturing real-life language demands, developers did not operationalize the targeted demands effectively. The two objectively scored parts of the test yielded unacceptable reliability coefficients. Internal consistency of the subjectively scored parts was spuriously high due to a halo effect and absence of explicit scoring benchmarks. A moderately high correlation between aural and reading parts was found. One analysis found that examinees who grew up speaking Spanish and spoke it currently in the home performed no better than those lacking these experiences. Content identified for the test was not fully embedded, what was incorporated was being used for the wrong grade levels, and it was skewed toward vocabulary, spelling, and grammar. It is concluded that making valid inferences concerning the language abilities of the examinees based on test scores is difficult, the social consequences of using pass-fail scores are undesirable, and the test is not adequately filling its intended purpose. (MSE)