Listeners rapidly adjust to talkers' pronunciations, accommodating those pronunciations into the relevant phonemic category to improve subsequent perception. Previous work has suggested that such learning is restricted to pronunciations that are representative of how the speaker talks (Kraljic, Samuel, & Brennan, 2008). If an ambiguous pronunciation, for example, can be attributed to an external source (such as a pen in the speaker's mouth), or if it is preceded by normal pronunciations of the same sound, learning is blocked. In three experiments, we explore this blocking effect in more detail. Our aim is to better understand the nature of the representations underlying the perceptual learning process. Experiment 1 replicates the blocking effect. Experiments 2 and 3 demonstrate that it can be eliminated when certain visual information occurs simultaneously with the auditory signal. The pattern of learning and non-learning is best accounted for by the view that speech perception is mediated by episodic representations that include potentially relevant visual information. (Contains 1 table and 3 figures.)