A sixth-grade mathematics test was evaluated in two different ways, and the evaluations were compared. One method of evaluation was to use the Rasch model for item analysis and to accept or reject items on the basis of the extent to which the data fit the model. The other method of evaluation was to ask school superintendents and teachers to rate the items and objectives in terms of their appropriateness for the students and the curriculum. The Rasch model item analysis rejected fifteen of the forty test items and a factor analysis of the test indicated that the rejected items were not measures of the first factor measured by the test. There was better than chance agreement between the item analysis and the ratings by teachers or superintendents. Very few items were accepted by the item analysis criterion and rejected by the rating criteria. It was concluded that the best evaluation of the items would include both criteria. (CTM)