In this paper, the multilateral comparison method of classifying languages is described and analyzed. It is suggested that while it is espoused as a simple and reasonable approach to language classification, the method has serious flaws. "Multilateral" or "mass" comparison (MC) is not a method of genetic language classification, but a pre-theoretical step preceding comparative/historical reconstruction. MC classification of languages into different families is based on discovering words in different languages that are similar in sound and meaning. Refutation of the method is based largely on statistical analysis of the probabilities of arriving at correct answers, using a test found reliable in several previous studies. In this case it is applied to a genetic classification of African languages. Criticisms focus on errors, uneven documentation, and data not supporting the findings. Several reasons are offered for the good results obtained despite flawed data and analysis, with examples provided from East Sudanic and Nilo-Saharan analyses. Contains 17 references. (MSE)