This study aims to examine the relationship between vocabulary proficiency and short-term memory capacity in deaf or hard-of-hearing (DHH) children. We test the hypothesis that the relationship between vocabulary skills and digit span performance could be strengthened when the digit span task encompasses cross-modal integration processes. A group of DHH children performed two types of auditory digit span tasks. Furthermore, they participated in a standardized vocabulary proficiency test, comprising two subtests: Receptive Vocabulary and Expressive Vocabulary. The verbal digit span served as a significant predictor of Expressive Vocabulary among the DHH children. Simultaneously, the auditory-pointing digit span accounted for a substantial portion of performance variation in both Receptive and Expressive Vocabulary. After considering the impact of the duration of auditory-verbal intervention through regression models, likelihood ratio tests demonstrated that the auditory-pointing digit span persisted as a significant determinant of both receptive and expressive vocabulary skills. A positive influence of the intervention was also confirmed by the present results. This study provides evidence that memory span and the ability to integrate cross-modal information could serve as significant cognitive correlates of vocabulary proficiency in DHH children.