To better understand the relationship between second-language (L2) listening comprehension and metacognitive awareness, we examined the moderating effects of listening tests and learner samples while focusing on aspects of metacognitive awareness. Students of English-as-a-foreign-language at a Japanese university (n = 75; the 2019 cohort) took the Test of English as a Foreign Language Institutional Testing Program (TOEFL ITP®) test, a paper-based TOEFL; the TOEFL Internet-based test (TOEFL iBT®); and the Metacognitive Awareness Listening Questionnaire (MALQ). Another group of students (n = 107; the 2020 cohort) took the TOEFL ITP and MALQ. Random forest analysis was applied to the results of the 2019 cohort, showing that, in order of importance, person knowledge, mental translation, and directed attention were related to listening comprehension in both listening tests. Problem solving was not related in either listening test. Further, planning and evaluation strategies were related to listening comprehension only in the TOEFL ITP. Comparison between the TOEFL ITP results of the 2019 and 2020 cohorts showed that only person knowledge was related to listening comprehension across the two cohorts, indicating a strong generalizability of person knowledge and weak generalizability of the remaining metacognitive strategies across learners. Implications and future directions are discussed.