Perspective-taking plays an important role in daily life, allowing consideration of other people's perspectives and viewpoints. This study used a large sample of 265 community-based participants (aged 20-86 years) to examine changes in perspective-taking abilities--a component of "Theory of Mind"--across adulthood, and how these changes may relate to individual differences in executive functions at different ages. Participants completed a referential-communication task (the "Director" task) while behavioral responses and eye movements were recorded, along with four measures of executive functions (inhibitory control, working memory, cognitive flexibility, and planning). Results revealed a quadratic fit of age in egocentric errors; performance on the task plateaued between 20 to [approximately]37 years old but showed a substantial decline from [approximately]38 years onward (i.e., increased egocentric errors). A similar pattern was established in eye-movement measures, demonstrating that advancing age led to a decrease in efficient attention orientation to a target. In other words, older adults were more distracted by a hidden competitor object (egocentric interference) and were therefore delayed in orienting their attention to the correct target object. Mediation analyses revealed that executive functions partially mediated the effect of age on perspective-taking abilities. Importantly, however, the relationship between age and egocentric bias in task performance remained significant when controlling for changes in executive functions, indicating a decline in social cognition abilities with advancing age that was independent of age-related declines in more domain-general abilities, such as executive functions.