The current study examined the phonolexical processing of Mandarin segments and tones by English speakers at different Mandarin proficiency levels. Eleven English speakers naive to Mandarin, 15 intermediate and 9 advanced second language (L2) learners participated in a word-learning experiment. After learning the sound and meaning of 16 Mandarin disyllabic words, they judged the matching between sound and meaning pairs, with half of the pairs being complete matches while the other half contained segmental or tonal mismatches. The results showed that all three groups were more sensitive to segmental than tonal mismatches. The two learner groups outperformed the Naive group on segmental mismatches but not on tonal mismatches. However, their reaction times revealed that the learners but not the Naive group attended to tonal variations. The current findings suggest that increasing L2 experience has limited benefit on learners' phonolexical processing of L2 tones, probably due to their non-tonal native language background. Experience in a tonal L2 may enhance learners' attention to the tonal dimension but may not necessarily improve their accuracy.