The decision by the National Institute of Education panel on the effects of school desegregation to select (for meta-analysis) a small group of preferred studies based upon criteria chosen in advance of examining the studies was, in principle, a mistake. One usually cannot know until the data have been examined which of several competing methodological criteria are most important. In the case of the effects of desegregation on minority achievement, Crain and Mahard in their 1982 review of 93 desegregation studies found a methodological error so specific to desegregation research that it was not even recognized as an error until the review was done. The error was that studies of the effects of desegregation on minority achievement will underestimate any effects when using subjects who have not been in desegregated settings since kindergarten or Grade 1. Whereas Crain and Mahard found 20 studies of blacks in desegregated settings since kindergarten or Grade 1, the panel discarded all but one of them because they did not fit their chosen-in-advance criteria. Of the 20 studies identified by Crain and Mahard, 16 showed consistent positive outcomes and only 2 were negative. If the principal function of selecting a superior subgroup of studies is to find the consistency of results which is masked by error in an unselected sample, Crain and Mahard succeeded, and the panel did not. (CMG)