Spoken words vary phonetically along a number of dimensions, such as duration, pitch, and vowel quality. Much of this variation is associated with social factors like the dialect, age, or gender of the speaker -- a type of variation termed 'socio-indexical'. Traditional theories of speech perception have seen this socio-indexical variation as a source of noise that listeners must 'filter out', in order to match the noisy speech signal to abstract mental representations of words or phonemes. But more recent theories propose that socially salient variation can actually make speech perception easier for listeners, as long as the variation is congruent with their experience (exemplar theory: Goldinger, 1996; Johnson, 2006), because listeners maintain specific detailed memories of instances of language that they have experienced. Exemplar models make fairly narrow predictions about the interactions between specific memories and socio-indexical information -- specific memories are represented only at the surface phonetic level, so they should not affect deeper linguistic systems like semantics; and socio-indexical group recognition is a byproduct of word recognition, happening serially after words are recognized. Recent evidence, however, raises significant questions about these predictions. In this dissertation, I argue that while socio-indexical variation does facilitate word recognition, this facilitation is more widespread than expected. I begin by using several large collections of spoken conversations to determine which words are used more often by women and which more often by men. I then use laboratory experiments to establish that listeners' sensitivity to these group-specific word frequencies interacts with both phonology and semantics beyond the predictions of exemplar theory, suggesting the need for more complex spoken word recognition mechanisms. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]