Categorical perception of color is shown when colors from the same category are discriminated less easily than equivalently spaced colors that cross a category boundary. The current experiments tested various models of categorical perception. Experiment 1 tested for categorical responding in 2- to 4-year-olds, the age range for the onset establishment of color term knowledge. Experiment 2 tested for categorical responding in Himba toddlers, whose language segments the color space differently from the way in which the English language does so. Experiment 3 manipulated the conditions of the task to explore whether the categorical responding in Experiments 1 and 2 was equivalent to categorical perception. Categorical perception was shown irrespective of naming and was not stronger in those children with more developed color term knowledge. Cross-cultural differences in the extent of categorical perception were not found. These findings support universalistic models of color categorization and suggest that color term knowledge does not modify categorical perception, at least during the early stages of childhood.