In order to address the problem of the underlying mechanism of mathematics learning disability (MLD), in the current research, we investigated the visual and auditory perceptions of elementary school children with visual and auditory MLD from the first to fourth grade. We found that children with visual mathematics learning disability (VMLD) and those with auditory mathematics learning disability (AMLD) all had problems in both visual and auditory perceptions. After further analysis, we found that children with VMLD suffered from more serious problems with visual perception than with auditory perception while it was the other way around for children with AMLD. Children with visual and auditory mathematics learning disability (V-AMLD) had both serious problems with visual and auditory perceptions. It thus can be seen that the visual Arabic code of number was more closely tied with the visual perception while the auditory-verbal code of number more closely associated with the auditory perception. The results shed light on the subtypes of MLD and contributed to the intervention.