Purpose: The purpose of this study was to investigate whether speech perception would reflect small latency changes in subcortical speech representation. Method: Twelve native Japanese listeners participated in the experiment. Those listeners participated in speech identification task and auditory brainstem response (ABR) measurement using /d/--/t/ continuum stimuli varying in voice onset time (VOT) with manipulation of the amplitude of initial noise (consonant) portion, the duration of which corresponded to VOT. Results: Increasing the noise portion amplitude lengthened subcortical representation of VOT, which is the latency difference between ABRs synchronizing to the onsets of initial noise and following periodic (vowel) portions (VOT ABR) and made listeners likely to perceive the stimuli with ambiguous VOT as a voiceless stop /t/. In addition, the amount of VOT ABR lengthening was close to that of the VOT boundary shortening. Conclusion: A few milliseconds of difference in subcortical speech representation are important for the perception of speech sounds with ambiguous acoustic cues.