In a changing society where the roles of fathers and mothers in caregiving are becoming more equal, the role of the father in early language development has also changed. We aimed to study associations between paternal factors and early vocabulary development in boys and girls. In a longitudinal cohort study, we examined the growth of expressive vocabulary in 354 boys and 331 girls between 13 and 24 months of age using the MacArthur-Bates Communicative Development Inventory. The results show that boys who had fathers not working full time, who had mothers with higher occupational status, and who had a larger vocabulary size at 13 months of age had larger gains in vocabulary. Girls with fathers working as professionals (high occupational status) had larger vocabulary growth compared to girls with fathers of lower occupational status. The results demonstrate that vocabulary growth in boys and girls relates differently to environmental factors. The results highlight the importance of further studies on fathers' role in children's early vocabulary development and the need to analyse the influence of environmental factors on early language development as a function of the child's sex.