A growing body of work suggests that speaker-race influences how infants and toddlers interpret the meanings of words. In two experiments, we explored the role of speaker-race on whether newly learned word-object pairs are generalized to new speakers. Seventy-two 20-month-olds were taught two word-object pairs from a familiar race speaker, and two different word-object pairs from an unfamiliar race speaker (four new pairs total). Using an intermodal preferential looking procedure, their interpretation of these new word-object pairs was tested using an unpictured novel speaker. We found that toddlers did not generalize word meanings taught by an unfamiliar race speaker to a new speaker (Experiment 1), unless given evidence that the unfamiliar race speaker was a member of the child's linguistic community through affiliative behaviour and linguistic competence (Experiment 2). In both experiments, generalization was observed for the word-object pairs taught by the familiar race speaker. These experiments indicate that children attend to speakers' non-linguistic properties, and this, in turn, can influence the perceived relevance of speakers' labels.