Perspective-taking, which is important for communication and social activities, can be cultivated through joint actions, including musical activities in children. We examined how rhythmic activities requiring coordination affect perspective-taking in a referential communication task with 100 Chinese 4- to 6-year-old children. In Study 1, 5- to 6-year-old children played an instrument with a virtual partner in one of three coordination conditions: synchrony, asynchrony, and antiphase synchrony. Eye movements were then monitored with the partner giving instructions to identify a shape referent which included a pre-nominal scalar adjective (e.g., big cubic block). When the target contrast (a small cubic block) was in the shared ground and a competitor contrast was occluded for the partner, participants who used perspective differences could, in principle, identify the intended referent before the shape was named. We hypothesized that asynchronous and antiphase synchronous musical activities, which require self-other distinction, might have stronger effects on perspective-taking than synchronous activity. Children in the asynchrony and antiphase synchrony conditions, but not the synchrony condition, showed anticipatory looks at the target, demonstrating real-time use of the partner's perspective. Study 2 was conducted to determine if asynchrony and antiphase asynchrony resulted in perspective-taking that otherwise would not have been observed, or if synchronous coordination inhibited perspective-taking that would otherwise have occurred. We found no evidence for online perspective-taking in 4- to 6-year-old children without music manipulation. Therefore, playing instruments asynchronously or in alternation, but not synchronously, increases perspective-taking in children of this age, likely by training self-other distinction and control. A video abstract of this article can be viewed at https://youtu.be/TM9h_GpFlsA.