This study is the first attempt to explore the relationship between rater variables focusing on raters' language aptitude and their judgments of second language (L2) speech. Thirty-four English listeners rated 65 spontaneous native and nonnative speech samples for comprehensibility, accentedness, and fluency. They also completed the LLAMA language aptitude test for associate memory, phonetic memory, sound-symbol associations, and grammatical inferencing. The results of mixed effect models showed that native listeners' individual differences (i.e., language aptitude, major, accent familiarity and attitudes) altogether explained 10-12% of L2 speech ratings. A particularly novel finding is that all aptitude components exhibited significant effects on the ratings. Specifically, associative memory exhibited a positive association whereas and phonetic memory, sound-symbol associations, and grammatical inferencing had a negative association with the ratings. The findings have a potential to enhance our understanding of the complex dynamics involved in nonnative speech perception and L2 interaction, and implications for researchers and educators in SLA, language assessment, and education.