Humans excel in familiar face recognition, but often find it hard to make identity judgements of unfamiliar faces. Understanding of the factors underlying the substantial benefits of familiarity is at present limited, but the effect is sometimes qualified by the way in which a face is known--for example, personal acquaintance sometimes gives rise to stronger familiarity effects than exposure through the media. Given the different quality of personal versus media knowledge, for example in one's emotional response or level of interaction, some have suggested qualitative differences between representations of people known personally or from media exposure. Alternatively, observed differences could reflect quantitative differences in the level of familiarity. We present 4 experiments investigating potential contributory influences to face familiarity effects in which observers view pictures showing their friends, favorite celebrities, celebrities they dislike, celebrities about whom they have expressed no opinion, and their own face. Using event-related potential indices with high temporal resolution and multiple highly varied everyday ambient images as a strong test of face recognition, we focus on the N250 and the later Sustained Familiarity Effect (SFE). All known faces show qualitatively similar responses relative to unfamiliar faces. Regardless of personal- or media-based familiarity, N250 reflects robust visual representations, successively refined over increasing exposure, while SFE appears to reflect the amount of identity-specific semantic information known about a person. These modulations of visual and semantic representations are consistent with face recognition models which emphasize the degree of familiarity but do not distinguish between different types of familiarity.