When schools are compared, there are variables that need to be considered because the playing field is not level. This paper discusses concepts and generalizations that must be understood to interpret test results and school report cards properly. There are many limitations in using test results to compare schools, particularly since students may be tested once a year at the most. The "one size fits all" approach of standardized testing fails to give a true picture of student learning and school effectiveness. Report cards for schools frequently omit important information such as expenditures, resources, and the condition of the school facilities. There are many problems in comparing longitudinal groups of students that may result from characteristics of the tests taken in different years or from differences in student characteristics. New assessment techniques, including portfolios, offer promise for providing data about schools, but care must be taken in comparisons based on qualitative data as well as comparisons based on quantitative data. (Contains 10 references.) (SLD)