High-quality caregiver-child relationships are critical to healthy development, including the acquisition of language and literacy skills. However, their potential benefits to early mathematical development and its underlying mechanisms have been minimally explored. In view that language and literacy development play prominent roles in mathematical development, this study investigated whether very young children's relationship quality with parents and teachers were linked with their mathematical ability through language and literacy abilities. One hundred and thirty two-year-olds from pre-nursery classes in Hong Kong were tested on their early academic abilities. Furthermore, their parents and teachers evaluated on questionnaires the quality of their relationships with the children. Results of path analyses indicated that after controlling for children's age, parent-child closeness was associated with children's mathematical ability via language ability. Teacher-child closeness had a direct link to children's mathematical ability, as well as indirect links through language and literacy abilities. Parent-child conflict and teacher-child conflict had neither direct nor indirect relations with children's mathematical ability. These results suggest that mathematical development is interwoven with language and literacy development beginning in the early years. Also, parents and teachers should be reminded the importance of building close and affectionate relationships with very young children when nurturing the foundation for academic development.