Current theories of Autism Spectrum Disorder (ASD) suggest atypical use of context in ASD, but little is known about how these atypicalities influence speech perception. We examined the influence of contextual information (lexical, spectral, and temporal) on phoneme categorization of people with ASD and in typically developed (TD) people. Across three experiments, we found that people with ASD used all types of contextual information for disambiguating speech sounds to the same extent as TD; yet they exhibited a shallower identification curve when phoneme categorization required temporal processing. Overall, the results suggest that the observed atypicalities in speech perception in ASD, including the reduced sensitivity observed here, cannot be attributed merely to the limited ability to utilize context during speech perception.