People's expectations help them make judgments about the world. In the area of spatial memory, the interaction of existing knowledge with incoming information is best illustrated in the category effect, a bias in positioning a target toward the prototypical location of its region (Huttenlocher et al., 1991). According to Bayesian principles, these biased judgments are weighted averages of a target's metric code (incoming data) and spatial categorical code (prior expectation). While previous research in the category effect generally focused on presence and reliability of different sources of information, we examined a scenario in which prior expectation is violated. Specifically, we investigated the role of prior expectations in location memory when these conflict with visual perception. We found that people favored their previous knowledge about where a target ought to be over the visual data on the actual target location. Hence, our work contributes to the literature by demonstrating the dominance of prior expectations over incongruent visual cues, and the data extend the context effect to spatial memory.