Scholars have argued for the inclusion of different spoken varieties of English in high-stakes listening tests to better represent the global use of English. However, doing so may introduce additional construct-irrelevant variance due to accent familiarity and the shared first language (L1) advantage, which could threaten test fairness. However, it is unclear to what extent accent familiarity and a shared L1 are related to or conflated with each other. The present study investigates the relationship between accent familiarity, a shared L1, and comprehensibility. Results from descriptive statistics and Mann-Whitney U test based on 302 second language (L2) English listeners' responses to an online questionnaire suggested that a shared L1 meant high accent familiarity, but not vice versa. A path analysis revealed a complex relationship between accent familiarity, a shared L1, and comprehensibility. While a shared L1 had a direct effect on accent familiarity, and accent familiarity had a direct effect on comprehensibility, a shared L1 did not predict comprehensibility when accent familiarity was controlled for. These results disentangle accent familiarity from a shared L1. Researchers should consider both constructs when investigating fairness in relation to World Englishes for listening assessment.